[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:00:52 dist-test-slave-cbsf krb5kdc[9387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:00:52 dist-test-slave-cbsf krb5kdc[9387](info): set up 2 sockets
Aug 11 09:00:52 dist-test-slave-cbsf krb5kdc[9387](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 09:00:53.711158 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:00:53 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:00:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:00:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:00:53.904596 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35965
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35965
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:54.247229 9402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:54.247535 9402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:54.247702 9402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:54.255865 9402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:00:54.255975 9402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:54.256024 9402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:54.256083 9402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:00:54.256127 9402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:00:54.266498 9402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:54.268113 9402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:54.269704 9402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:54.282572 9408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:54.282636 9412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:54.282613 9409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:54.283815 9402 server_base.cc:1034] running on GCE node
I20260811 09:00:54.284668 9402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:54.286255 9402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:54.287473 9402 hybrid_clock.cc:648] HybridClock initialized: now 1786438854287436 us; error 78 us; skew 500 ppm
I20260811 09:00:54.287842 9402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:54 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:54.291110 9402 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:00:54.292840 9402 webserver.cc:460] Webserver started at http://127.9.40.126:42429/ using document root <none> and password file <none>
I20260811 09:00:54.293560 9402 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:54.293715 9402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:54.294035 9402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:54.297406 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "2089720189c442f8a25605977a24c870"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "3f97a20335de778f46ae75ef87b2127a"
server_key_iv: "cc31f421c7014d4515e98a25bab02ca2"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.298223 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2089720189c442f8a25605977a24c870"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "3f97a20335de778f46ae75ef87b2127a"
server_key_iv: "cc31f421c7014d4515e98a25bab02ca2"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.304041 9402 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:00:54.308332 9420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:54.309836 9402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:00:54.310084 9402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2089720189c442f8a25605977a24c870"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "3f97a20335de778f46ae75ef87b2127a"
server_key_iv: "cc31f421c7014d4515e98a25bab02ca2"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.310309 9402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:54.340241 9402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:54.349308 9402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:54.349699 9402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:54.371460 9402 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35965
I20260811 09:00:54.371531 9476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35965 every 8 connection(s)
I20260811 09:00:54.373124 9402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:00:54.374900 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 9402
I20260811 09:00:54.375242 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:00:54.377406 9377 external_mini_cluster.cc:1428] Setting key 15bd88291ff45da56c845fc5ad983850
I20260811 09:00:54.377435 9478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:00:54.389750 9478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870: Bootstrap starting.
I20260811 09:00:54.393097 9478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870: Neither blocks nor log segments found. Creating new log.
I20260811 09:00:54.394284 9478 log.cc:824] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870: Log is configured to *not* fsync() on all Append() calls
I20260811 09:00:54.397280 9478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870: No bootstrap required, opened a new log
Aug 11 09:00:54 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:54.404865 9478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } }
I20260811 09:00:54.405256 9478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:00:54.405370 9478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2089720189c442f8a25605977a24c870, State: Initialized, Role: FOLLOWER
I20260811 09:00:54.406021 9478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } }
I20260811 09:00:54.406284 9478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:00:54.406445 9478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:00:54.406725 9478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:00:54.409579 9478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } }
I20260811 09:00:54.410135 9478 leader_election.cc:302] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2089720189c442f8a25605977a24c870; no voters:
I20260811 09:00:54.410880 9478 leader_election.cc:288] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:00:54.411015 9483 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:00:54.412372 9483 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [term 1 LEADER]: Becoming Leader. State: Replica: 2089720189c442f8a25605977a24c870, State: Running, Role: LEADER
I20260811 09:00:54.413033 9483 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } }
I20260811 09:00:54.413271 9478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:00:54.417624 9480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:54.381424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.9.40.126:35965)
0811 09:00:54.382481 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:54.382557 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:54.383650 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 09:00:54.392801 (+ 9151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:54.392824 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:54.393198 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:54.393666 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:54.396621 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:54.398154 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:54.399854 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:54.400565 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:54.400790 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:54.406735 (+ 5945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:54.406747 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:54.406763 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:54.406804 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:54.410090 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:54.411225 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:54.411236 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:54.411246 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:54.411583 (+ 337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:54.412363 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:54.412370 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:54.412373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:54.413285 (+ 912us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:54.413668 (+ 383us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:54.414893 (+ 1225us) server_negotiation.cc:299] Negotiation successful
0811 09:00:54.415613 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":553,"threads_started":1}
I20260811 09:00:54.422222 9484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2089720189c442f8a25605977a24c870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } } }
I20260811 09:00:54.422719 9484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [sys.catalog]: This master's current role is: LEADER
I20260811 09:00:54.422605 9485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2089720189c442f8a25605977a24c870. Latest consensus state: current_term: 1 leader_uuid: "2089720189c442f8a25605977a24c870" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2089720189c442f8a25605977a24c870" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35965 } } }
I20260811 09:00:54.423331 9485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870 [sys.catalog]: This master's current role is: LEADER
I20260811 09:00:54.424937 9496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:00:54.432487 9496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:00:54.443158 9496 catalog_manager.cc:1347] Generated new cluster ID: d28d75e4754e44348ed128320c5905de
I20260811 09:00:54.443324 9496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:00:54.461971 9496 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:00:54.463601 9496 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:00:54.471746 9496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2089720189c442f8a25605977a24c870: Generated new TSK 0
I20260811 09:00:54.472575 9496 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:00:54.544440 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:54.921527 9508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:54.921805 9508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:54.921972 9508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:54.929605 9508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:54.929710 9508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:54.929821 9508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:00:54.939754 9508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:54.941083 9508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:54.942380 9508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:54.956096 9520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:54.956207 9516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:54.956149 9517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:54.957373 9508 server_base.cc:1034] running on GCE node
I20260811 09:00:54.958035 9508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:54.959033 9508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:54.960254 9508 hybrid_clock.cc:648] HybridClock initialized: now 1786438854960215 us; error 33 us; skew 500 ppm
I20260811 09:00:54.960637 9508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:54 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:54.964026 9508 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:00:54.965669 9508 webserver.cc:460] Webserver started at http://127.9.40.65:35743/ using document root <none> and password file <none>
I20260811 09:00:54.966339 9508 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:54.966441 9508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:54.966724 9508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:54.969770 9508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "fab1688622f64c468c5c5e08efefe7be"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "a8c4f652ea238b87cb216a5b6c2957d3"
server_key_iv: "3f78f8b91de2679b51001c0ba2cdc393"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.970624 9508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "fab1688622f64c468c5c5e08efefe7be"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "a8c4f652ea238b87cb216a5b6c2957d3"
server_key_iv: "3f78f8b91de2679b51001c0ba2cdc393"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.976082 9508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:00:54.979732 9528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:54.981055 9508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:00:54.981226 9508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "fab1688622f64c468c5c5e08efefe7be"
format_stamp: "Formatted at 2026-08-11 09:00:54 on dist-test-slave-cbsf"
server_key: "a8c4f652ea238b87cb216a5b6c2957d3"
server_key_iv: "3f78f8b91de2679b51001c0ba2cdc393"
server_key_version: "encryptionkey@0"
I20260811 09:00:54.981396 9508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:54.993525 9508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:54.999289 9508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:54.999620 9508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:55.000877 9508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:00:55.002272 9508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:00:55.002363 9508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:55.002506 9508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:00:55.002593 9508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:55.040436 9508 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45729
I20260811 09:00:55.040529 9660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45729 every 8 connection(s)
I20260811 09:00:55.041808 9508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:00:55.050307 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 9508
I20260811 09:00:55.050565 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:00:55.051302 9377 external_mini_cluster.cc:1428] Setting key 82eedc78c009a1ade10b407146037df9
Aug 11 09:00:55 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438854, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:55.063164 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:55.044680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51233 (local address 127.9.40.126:35965)
0811 09:00:55.045188 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:55.045192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:55.046175 (+ 983us) server_negotiation.cc:407] Connection header received
0811 09:00:55.047944 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:55.047949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:55.048013 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:55.048110 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:55.049664 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.050831 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.052648 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.053124 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.053263 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:55.057195 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:55.057200 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:55.057204 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:55.057234 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:55.059887 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:55.060676 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:55.060680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:55.060682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:55.060888 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:55.061354 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:55.061358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:55.061379 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:55.061693 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:55.061812 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:55.062722 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 09:00:55.062890 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":236,"threads_started":1}
I20260811 09:00:55.064952 9669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:55.045266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35965 (local address 127.9.40.65:51233)
0811 09:00:55.046002 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 09:00:55.046042 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:00:55.047688 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:00:55.048309 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:00:55.048317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:00:55.048929 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:00:55.049499 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:55.049516 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.050978 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.050982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:55.052502 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:55.052524 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.053352 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.053360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:55.053624 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:55.054315 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:00:55.054337 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:00:55.056970 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:00:55.060055 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:55.060063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:55.060080 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:55.060528 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:55.061006 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:55.061011 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:55.061014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:55.061207 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:55.061828 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:00:55.061839 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:00:55.062314 (+ 475us) client_negotiation.cc:769] Sending connection context
0811 09:00:55.062628 (+ 314us) client_negotiation.cc:240] Negotiation successful
0811 09:00:55.063235 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":301,"threads_started":1}
I20260811 09:00:55.066609 9664 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35965
I20260811 09:00:55.066946 9664 heartbeater.cc:464] Registering TS with master...
I20260811 09:00:55.067693 9664 heartbeater.cc:511] Master 127.9.40.126:35965 requested a full tablet report, sending...
I20260811 09:00:55.069977 9439 ts_manager.cc:194] Registered new tserver with Master: fab1688622f64c468c5c5e08efefe7be (127.9.40.65:45729)
I20260811 09:00:55.071475 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:51233
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:00:55.097894 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:55.441205 9672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:55.441529 9672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:55.441633 9672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:55.448792 9672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:55.448895 9672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:55.449014 9672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:00:55.458572 9672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:55.459785 9672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:55.461103 9672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:55.474579 9678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:55.474589 9682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:55.474686 9677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:55.475584 9672 server_base.cc:1034] running on GCE node
I20260811 09:00:55.476234 9672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:55.477303 9672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:55.478536 9672 hybrid_clock.cc:648] HybridClock initialized: now 1786438855478489 us; error 64 us; skew 500 ppm
I20260811 09:00:55.478847 9672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:55 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:55.482213 9672 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:00:55.483655 9672 webserver.cc:460] Webserver started at http://127.9.40.66:33745/ using document root <none> and password file <none>
I20260811 09:00:55.484428 9672 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:55.484587 9672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:55.484987 9672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:55.487946 9672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "243d43272bbf4dc68f17992b8078b103"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "8fda8edb27513005a9e587af117e0248"
server_key_iv: "a7faceda4b60796cf946cbbe17ea993c"
server_key_version: "encryptionkey@0"
I20260811 09:00:55.488816 9672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "243d43272bbf4dc68f17992b8078b103"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "8fda8edb27513005a9e587af117e0248"
server_key_iv: "a7faceda4b60796cf946cbbe17ea993c"
server_key_version: "encryptionkey@0"
I20260811 09:00:55.493886 9672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:00:55.497386 9689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:55.498668 9672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:00:55.498848 9672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "243d43272bbf4dc68f17992b8078b103"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "8fda8edb27513005a9e587af117e0248"
server_key_iv: "a7faceda4b60796cf946cbbe17ea993c"
server_key_version: "encryptionkey@0"
I20260811 09:00:55.499048 9672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:55.513944 9672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:55.518100 9672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:55.518409 9672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:55.519368 9672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:00:55.520787 9672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:00:55.520893 9672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:55.521050 9672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:00:55.521130 9672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:55.558485 9672 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:33701
I20260811 09:00:55.558570 9822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:33701 every 8 connection(s)
I20260811 09:00:55.559818 9672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:00:55.565944 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 9672
I20260811 09:00:55.566224 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:00:55.566816 9377 external_mini_cluster.cc:1428] Setting key a5f0a4f10d7b1a2f83cfad853b542862
Aug 11 09:00:55 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:55.583957 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:55.562446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35019 (local address 127.9.40.126:35965)
0811 09:00:55.562641 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:55.562646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:55.563791 (+ 1145us) server_negotiation.cc:407] Connection header received
0811 09:00:55.565821 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:55.565828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:55.565971 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:55.566158 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:55.568291 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.569117 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.571936 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.572357 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.572452 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:55.577110 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:55.577120 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:55.577127 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:55.577184 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:55.580359 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:55.581451 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:55.581455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:55.581457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:55.581594 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:55.582100 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:55.582103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:55.582105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:55.582365 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:55.582505 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:55.583544 (+ 1039us) server_negotiation.cc:299] Negotiation successful
0811 09:00:55.583717 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:00:55.585659 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:55.562860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35965 (local address 127.9.40.66:35019)
0811 09:00:55.563609 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 09:00:55.563655 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:00:55.565528 (+ 1873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:00:55.566510 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:00:55.566522 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:00:55.567339 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:00:55.568093 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:55.568119 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.569277 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.569282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:55.571770 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:55.571791 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:55.572673 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:55.572683 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:55.573140 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:55.574013 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:00:55.574050 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:00:55.576850 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:00:55.580513 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:55.580542 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:55.580562 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:55.581297 (+ 735us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:55.581718 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:55.581725 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:55.581731 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:55.581984 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:55.582535 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:00:55.582546 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:00:55.583096 (+ 550us) client_negotiation.cc:769] Sending connection context
0811 09:00:55.583420 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 09:00:55.584020 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":307,"threads_started":1}
I20260811 09:00:55.587224 9823 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35965
I20260811 09:00:55.587579 9823 heartbeater.cc:464] Registering TS with master...
I20260811 09:00:55.588354 9823 heartbeater.cc:511] Master 127.9.40.126:35965 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260811 09:00:55.590072 9439 ts_manager.cc:194] Registered new tserver with Master: 243d43272bbf4dc68f17992b8078b103 (127.9.40.66:33701)
I20260811 09:00:55.590876 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:00:55.613857 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:55.951836 9828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:55.952126 9828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:55.952282 9828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:55.959560 9828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:55.959661 9828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:55.959875 9828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:00:55.969787 9828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35965
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:55.971061 9828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:55.972397 9828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:55.986100 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:55.986168 9837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:55.986202 9839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:55.987313 9828 server_base.cc:1034] running on GCE node
I20260811 09:00:55.987974 9828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:55.989070 9828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:55.990300 9828 hybrid_clock.cc:648] HybridClock initialized: now 1786438855990226 us; error 53 us; skew 500 ppm
I20260811 09:00:55.990659 9828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:55 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:55.994475 9828 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:00:55.996255 9828 webserver.cc:460] Webserver started at http://127.9.40.67:45297/ using document root <none> and password file <none>
I20260811 09:00:55.997082 9828 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:55.997224 9828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:55.997551 9828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:56.000636 9828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "bd42bbf4481e40279b8a91cb7b050632"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "199602050fa0ba5fcba125e6f9c726ea"
server_key_iv: "b2247fd4de9c3117084cb67bdfd9c4c5"
server_key_version: "encryptionkey@0"
I20260811 09:00:56.001650 9828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "bd42bbf4481e40279b8a91cb7b050632"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "199602050fa0ba5fcba125e6f9c726ea"
server_key_iv: "b2247fd4de9c3117084cb67bdfd9c4c5"
server_key_version: "encryptionkey@0"
I20260811 09:00:56.007428 9828 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:00:56.011390 9847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:56.012781 9828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:00:56.012971 9828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "bd42bbf4481e40279b8a91cb7b050632"
format_stamp: "Formatted at 2026-08-11 09:00:55 on dist-test-slave-cbsf"
server_key: "199602050fa0ba5fcba125e6f9c726ea"
server_key_iv: "b2247fd4de9c3117084cb67bdfd9c4c5"
server_key_version: "encryptionkey@0"
I20260811 09:00:56.013207 9828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:56.028807 9828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:56.032999 9828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:56.033313 9828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:56.034291 9828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:00:56.035835 9828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:00:56.035943 9828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:56.036109 9828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:00:56.036195 9828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:56.074618 9664 heartbeater.cc:503] Master 127.9.40.126:35965 was elected leader, sending a full tablet report...
I20260811 09:00:56.076645 9828 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:40573
I20260811 09:00:56.076666 9974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:40573 every 8 connection(s)
I20260811 09:00:56.077966 9828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:00:56.082752 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 9828
I20260811 09:00:56.082968 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:00:56.083540 9377 external_mini_cluster.cc:1428] Setting key 33bc282f258a9075e18b0fccd3ed0cc0
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438855, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:56.100481 9668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.080871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57455 (local address 127.9.40.126:35965)
0811 09:00:56.081053 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.081057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.082534 (+ 1477us) server_negotiation.cc:407] Connection header received
0811 09:00:56.085068 (+ 2534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.085073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.085186 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.085295 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.087259 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.088186 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.089861 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.090346 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.090517 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.094156 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.094161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.094164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.094199 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.096913 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.097656 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.097660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.097662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.097806 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.098539 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.098544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.098547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.098913 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.099064 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.099898 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.100124 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:00:56.101544 9977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.081340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35965 (local address 127.9.40.67:57455)
0811 09:00:56.082099 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 09:00:56.082163 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 09:00:56.084751 (+ 2588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:00:56.085619 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:00:56.085634 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:00:56.086495 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:00:56.087080 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:56.087098 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.088353 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.088359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:56.089694 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:56.089707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.090612 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.090622 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:56.090900 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.091569 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:00:56.091594 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:00:56.093951 (+ 2357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:00:56.097053 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:56.097059 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:56.097071 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:56.097532 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:56.097906 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:56.097909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:56.097912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:56.098188 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:56.099062 (+ 874us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:00:56.099073 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:00:56.099479 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 09:00:56.099814 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 09:00:56.100379 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":299,"threads_started":1}
I20260811 09:00:56.103073 9975 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35965
I20260811 09:00:56.103473 9975 heartbeater.cc:464] Registering TS with master...
I20260811 09:00:56.104205 9975 heartbeater.cc:511] Master 127.9.40.126:35965 requested a full tablet report, sending...
I20260811 09:00:56.105762 9439 ts_manager.cc:194] Registered new tserver with Master: bd42bbf4481e40279b8a91cb7b050632 (127.9.40.67:40573)
I20260811 09:00:56.106575 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:57455
I20260811 09:00:56.112404 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:00:56.163853 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.148107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.9.40.65:45729)
0811 09:00:56.148875 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.148883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.148922 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:00:56.148992 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.149000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.149246 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.149383 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.150730 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.151875 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.153366 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.153998 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.154107 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.158086 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.158098 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.158113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.158153 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.161145 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.161765 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.161773 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.161775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.161928 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.162308 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.162312 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.162314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.162698 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.162950 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.163232 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.163520 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":386,"threads_started":1}
W20260811 09:00:56.165408 9561 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49014
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:00:56.212203 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.198931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.9.40.65:45729)
0811 09:00:56.199198 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.199205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.199425 (+ 220us) server_negotiation.cc:407] Connection header received
0811 09:00:56.199577 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.199581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.199649 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.199732 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.201173 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.201877 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.203139 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.203573 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.203669 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.207527 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.207533 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.207536 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.207586 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.210057 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.210603 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.210606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.210609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.210725 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.211026 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.211029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.211031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.211262 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.211431 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.211671 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.211831 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 09:00:56.213897 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 9508
I20260811 09:00:56.226356 9982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.215068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49038 (local address 127.9.40.65:45729)
0811 09:00:56.215281 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.215287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.215461 (+ 174us) server_negotiation.cc:407] Connection header received
0811 09:00:56.215556 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.215560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.215614 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.215688 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.216840 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.217645 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.218909 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.219317 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.219410 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.221635 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.221644 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.221649 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.221684 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.224140 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.224748 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.224752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.224754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.224877 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.225185 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.225188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.225190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.225415 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.225591 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.225801 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.225942 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:00:56.227098 9656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:00:56.309986 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 9508
I20260811 09:00:56.330256 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 9672
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:00:56.347498 9989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.331234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.9.40.66:33701)
0811 09:00:56.331806 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.331813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.331840 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:00:56.331905 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.331911 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.332141 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.332266 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.333498 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.335122 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.337051 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.337628 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.337783 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.341813 (+ 4030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.341823 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.341837 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.341869 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.344576 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.345258 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.345265 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.345268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.345408 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.345823 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.345827 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.345829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.346189 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.346495 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.346870 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.347150 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":321,"threads_started":1}
I20260811 09:00:56.348788 9817 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:00:56.431628 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 9672
I20260811 09:00:56.451610 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 9828
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:00:56.467916 9991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:56.452794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58900 (local address 127.9.40.67:40573)
0811 09:00:56.453460 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:56.453467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:56.453497 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:00:56.453557 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:56.453565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:56.453777 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:56.453951 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:56.455233 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.456300 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.457790 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:56.458369 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:56.458519 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:56.462607 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:56.462619 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:56.462633 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:56.462664 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:56.465435 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.466160 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.466167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.466170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.466283 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:56.466615 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:56.466618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:56.466620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:56.466928 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:56.467176 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:56.467411 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 09:00:56.467599 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":341,"threads_started":1}
I20260811 09:00:56.469306 9968 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:00:56.551513 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 9828
I20260811 09:00:56.571494 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 9402
I20260811 09:00:56.572831 9472 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:00:56.674883 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 9402
2026-08-11T09:00:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4164 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9996](info): set up 2 sockets
Aug 11 09:00:56 dist-test-slave-cbsf krb5kdc[9996](info): commencing operation
krb5kdc: starting...
W20260811 09:00:57.907402 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:00:58 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:00:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:00:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:00:58.102586 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34091
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:58.419071 10012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:58.419390 10012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:58.419564 10012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:58.426640 10012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:00:58.426779 10012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:58.426887 10012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:58.426990 10012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:00:58.427071 10012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:00:58.436630 10012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:58.437963 10012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:58.439237 10012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:58.450736 10017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:58.450861 10018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:58.451095 10020 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:58.451892 10012 server_base.cc:1034] running on GCE node
I20260811 09:00:58.452656 10012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:58.454031 10012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:58.455257 10012 hybrid_clock.cc:648] HybridClock initialized: now 1786438858455209 us; error 32 us; skew 500 ppm
I20260811 09:00:58.455678 10012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:58 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438858, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:58.458984 10012 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:00:58.460381 10012 webserver.cc:460] Webserver started at http://127.9.40.126:39079/ using document root <none> and password file <none>
I20260811 09:00:58.461146 10012 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:58.461289 10012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:58.461625 10012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:58.464514 10012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "21353cd9f2254ac78e938ef762ffa398"
format_stamp: "Formatted at 2026-08-11 09:00:58 on dist-test-slave-cbsf"
server_key: "794e8b21a17b6dbc72f872d27e017a92"
server_key_iv: "5867504aae03a02283683e2a9698a85c"
server_key_version: "encryptionkey@0"
I20260811 09:00:58.465312 10012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "21353cd9f2254ac78e938ef762ffa398"
format_stamp: "Formatted at 2026-08-11 09:00:58 on dist-test-slave-cbsf"
server_key: "794e8b21a17b6dbc72f872d27e017a92"
server_key_iv: "5867504aae03a02283683e2a9698a85c"
server_key_version: "encryptionkey@0"
I20260811 09:00:58.470321 10012 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:00:58.473766 10029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:58.475045 10012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:00:58.475236 10012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "21353cd9f2254ac78e938ef762ffa398"
format_stamp: "Formatted at 2026-08-11 09:00:58 on dist-test-slave-cbsf"
server_key: "794e8b21a17b6dbc72f872d27e017a92"
server_key_iv: "5867504aae03a02283683e2a9698a85c"
server_key_version: "encryptionkey@0"
I20260811 09:00:58.475454 10012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:58.533551 10012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:58.537982 10012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:58.538287 10012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:58.556203 10012 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:34091
I20260811 09:00:58.556245 10086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:34091 every 8 connection(s)
I20260811 09:00:58.557634 10012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:00:58.558351 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10012
I20260811 09:00:58.558646 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:00:58.559484 9377 external_mini_cluster.cc:1428] Setting key 5364a10b8b51479658d258f8542b50b8
I20260811 09:00:58.562099 10087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:00:58 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:58.575004 10087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398: Bootstrap starting.
I20260811 09:00:58.578365 10087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398: Neither blocks nor log segments found. Creating new log.
I20260811 09:00:58.579444 10087 log.cc:824] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398: Log is configured to *not* fsync() on all Append() calls
I20260811 09:00:58.582286 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:58.561608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44660 (local address 127.9.40.126:34091)
0811 09:00:58.562439 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:58.562451 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:58.562489 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:00:58.564036 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:58.564057 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:58.564414 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:58.564801 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:58.566611 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:58.567950 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:58.569749 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:58.570368 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:58.570488 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:58.575169 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:58.575180 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:58.575203 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:58.575241 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:58.578140 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:58.578967 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:58.578973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:58.578979 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:58.579138 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:58.579581 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:58.579586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:58.579589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:58.580121 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:58.580441 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:58.580994 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 09:00:58.581356 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":607,"thread_start_us":373,"threads_started":1}
I20260811 09:00:58.582396 10087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398: No bootstrap required, opened a new log
I20260811 09:00:58.590003 10087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } }
I20260811 09:00:58.590399 10087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:00:58.590515 10087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21353cd9f2254ac78e938ef762ffa398, State: Initialized, Role: FOLLOWER
I20260811 09:00:58.591118 10087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } }
I20260811 09:00:58.591424 10087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:00:58.591570 10087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:00:58.591727 10087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:00:58.593751 10087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } }
I20260811 09:00:58.594152 10087 leader_election.cc:302] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21353cd9f2254ac78e938ef762ffa398; no voters:
I20260811 09:00:58.594645 10087 leader_election.cc:288] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:00:58.594796 10094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:00:58.595983 10094 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [term 1 LEADER]: Becoming Leader. State: Replica: 21353cd9f2254ac78e938ef762ffa398, State: Running, Role: LEADER
I20260811 09:00:58.596501 10094 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } }
I20260811 09:00:58.596735 10087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:00:58.599598 10095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21353cd9f2254ac78e938ef762ffa398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } } }
I20260811 09:00:58.599572 10096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21353cd9f2254ac78e938ef762ffa398. Latest consensus state: current_term: 1 leader_uuid: "21353cd9f2254ac78e938ef762ffa398" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21353cd9f2254ac78e938ef762ffa398" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34091 } } }
I20260811 09:00:58.600082 10096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [sys.catalog]: This master's current role is: LEADER
I20260811 09:00:58.600082 10095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398 [sys.catalog]: This master's current role is: LEADER
I20260811 09:00:58.601617 10104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:00:58.606825 10104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:00:58.615517 10104 catalog_manager.cc:1347] Generated new cluster ID: f6c60a795f214cbf98c7d87e3d268c3d
I20260811 09:00:58.615628 10104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:00:58.623144 10104 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:00:58.624320 10104 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:00:58.635540 10104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21353cd9f2254ac78e938ef762ffa398: Generated new TSK 0
I20260811 09:00:58.636253 10104 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:00:58.685319 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:59.017386 10123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:59.017665 10123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:59.017854 10123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:59.025063 10123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:59.025166 10123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:59.025346 10123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:00:59.034926 10123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:59.036262 10123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:59.037660 10123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:59.050716 10130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:59.050797 10129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:59.050693 10132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:59.051784 10123 server_base.cc:1034] running on GCE node
I20260811 09:00:59.052402 10123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:59.053373 10123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:59.054606 10123 hybrid_clock.cc:648] HybridClock initialized: now 1786438859054563 us; error 36 us; skew 500 ppm
I20260811 09:00:59.054924 10123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:59 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438859, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:59.058221 10123 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:00:59.059830 10123 webserver.cc:460] Webserver started at http://127.9.40.65:39615/ using document root <none> and password file <none>
I20260811 09:00:59.060525 10123 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:59.060657 10123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:59.061084 10123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:59.063915 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "a0a7d31491284289831b6e0a7e0e45bf"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "f5499c68f71f7eb9a86a14f3d05f3860"
server_key_iv: "b73d8e27f650e8ee542d72820695e06f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.064779 10123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "a0a7d31491284289831b6e0a7e0e45bf"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "f5499c68f71f7eb9a86a14f3d05f3860"
server_key_iv: "b73d8e27f650e8ee542d72820695e06f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.069885 10123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:00:59.073304 10139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.074762 10123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:00:59.075004 10123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "a0a7d31491284289831b6e0a7e0e45bf"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "f5499c68f71f7eb9a86a14f3d05f3860"
server_key_iv: "b73d8e27f650e8ee542d72820695e06f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.075237 10123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:59.096271 10123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:59.100641 10123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:59.101008 10123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:59.102193 10123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:00:59.103850 10123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:00:59.103974 10123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.104147 10123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:00:59.104228 10123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.141608 10123 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33771
I20260811 09:00:59.141644 10278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33771 every 8 connection(s)
I20260811 09:00:59.143011 10123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:00:59.144935 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10123
I20260811 09:00:59.145216 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:00:59.146145 9377 external_mini_cluster.cc:1428] Setting key df63b642dd35549382403ed9fa75124a
Aug 11 09:00:59 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438859, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:59.168100 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:59.145818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:50725 (local address 127.9.40.126:34091)
0811 09:00:59.146422 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:59.146429 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:59.147767 (+ 1338us) server_negotiation.cc:407] Connection header received
0811 09:00:59.150472 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:59.150476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:59.150547 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:59.150653 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:59.152828 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.153608 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.155847 (+ 2239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.156544 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.156749 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:59.160749 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:59.160759 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:59.160766 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:59.160811 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:59.164573 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:59.165444 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:59.165451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:59.165457 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:59.165624 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:59.166074 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:59.166082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:59.166088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:59.166467 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:59.166636 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:59.167348 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 09:00:59.167600 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":286,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:00:59.168833 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:59.146421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:34091 (local address 127.9.40.65:50725)
0811 09:00:59.147585 (+ 1164us) negotiation.cc:107] Waiting for socket to connect
0811 09:00:59.147638 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:00:59.150083 (+ 2445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:00:59.150924 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:00:59.150936 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:00:59.151778 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:00:59.152607 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:59.152635 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.153765 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.153773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:59.155684 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:59.155696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.156870 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.156875 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:59.157092 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:59.157864 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:00:59.157891 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:00:59.160462 (+ 2571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:00:59.164828 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:59.164835 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:59.164848 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:59.165288 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:59.165749 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:59.165752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:59.165754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:59.165942 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:59.166612 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:00:59.166618 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:00:59.166942 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 09:00:59.167177 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 09:00:59.167584 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"thread_start_us":689,"threads_started":1}
I20260811 09:00:59.170572 10279 heartbeater.cc:347] Connected to a master server at 127.9.40.126:34091
I20260811 09:00:59.170894 10279 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
I20260811 09:00:59.171718 10279 heartbeater.cc:511] Master 127.9.40.126:34091 requested a full tablet report, sending...
I20260811 09:00:59.173898 10050 ts_manager.cc:194] Registered new tserver with Master: a0a7d31491284289831b6e0a7e0e45bf (127.9.40.65:33771)
I20260811 09:00:59.175372 10050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:50725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:00:59.195806 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:00:59.524812 10286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:00:59.525117 10286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:00:59.525286 10286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:00:59.532258 10286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:00:59.532402 10286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:00:59.532595 10286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:00:59.542196 10286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:00:59.543481 10286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:00:59.544888 10286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:00:59.557678 10295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:59.557716 10292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:00:59.559105 10291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:00:59.559872 10286 server_base.cc:1034] running on GCE node
I20260811 09:00:59.560570 10286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:00:59.561574 10286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:00:59.562830 10286 hybrid_clock.cc:648] HybridClock initialized: now 1786438859562774 us; error 43 us; skew 500 ppm
I20260811 09:00:59.563187 10286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:00:59 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438859, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:00:59.567113 10286 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:00:59.570731 10286 webserver.cc:460] Webserver started at http://127.9.40.66:46305/ using document root <none> and password file <none>
I20260811 09:00:59.571616 10286 fs_manager.cc:359] Metadata directory not provided
I20260811 09:00:59.571756 10286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:00:59.572098 10286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:00:59.576045 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "e3d37a8ee9bb401ab4155c0421e1b910"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "eaede07ccb2f19dd7388f4bb55fcba5a"
server_key_iv: "a02ea802e64305114a13dd2dc1e96a9f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.577008 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "e3d37a8ee9bb401ab4155c0421e1b910"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "eaede07ccb2f19dd7388f4bb55fcba5a"
server_key_iv: "a02ea802e64305114a13dd2dc1e96a9f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.582720 10286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:00:59.586118 10304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.587385 10286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:00:59.587517 10286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "e3d37a8ee9bb401ab4155c0421e1b910"
format_stamp: "Formatted at 2026-08-11 09:00:59 on dist-test-slave-cbsf"
server_key: "eaede07ccb2f19dd7388f4bb55fcba5a"
server_key_iv: "a02ea802e64305114a13dd2dc1e96a9f"
server_key_version: "encryptionkey@0"
I20260811 09:00:59.587685 10286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:00:59.605644 10286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:00:59.609642 10286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:00:59.609870 10286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:00:59.610730 10286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:00:59.612073 10286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:00:59.612144 10286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.612229 10286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:00:59.612273 10286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:00:59.646714 10286 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:43605
I20260811 09:00:59.646759 10440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:43605 every 8 connection(s)
I20260811 09:00:59.647964 10286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:00:59.654570 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10286
I20260811 09:00:59.654788 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:00:59.655306 9377 external_mini_cluster.cc:1428] Setting key c0c7ca56e10533f759a2de917fd69070
Aug 11 09:00:59 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438859, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:00:59.669437 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:59.650580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39297 (local address 127.9.40.126:34091)
0811 09:00:59.650762 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:00:59.650766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:00:59.651949 (+ 1183us) server_negotiation.cc:407] Connection header received
0811 09:00:59.653786 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:00:59.653790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:00:59.653862 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:00:59.653954 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:00:59.655630 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.656600 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.658532 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.659327 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.659461 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:59.663058 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:00:59.663066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:00:59.663073 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:00:59.663136 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:00:59.665947 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:59.666877 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:59.666880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:59.666882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:59.667016 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:00:59.667565 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:00:59.667569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:00:59.667571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:00:59.667834 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:00:59.667950 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:00:59.668927 (+ 977us) server_negotiation.cc:299] Negotiation successful
0811 09:00:59.669145 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:00:59.670946 10444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:00:59.651073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:34091 (local address 127.9.40.66:39297)
0811 09:00:59.651777 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 09:00:59.651814 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:00:59.653534 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:00:59.654150 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:00:59.654158 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:00:59.654681 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:00:59.655448 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:59.655471 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.656803 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.656809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:59.658389 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:00:59.658401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:00:59.659581 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:00:59.659587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:00:59.659795 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:00:59.660570 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:00:59.660615 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:00:59.662833 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:00:59.666086 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:59.666098 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:59.666117 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:59.666760 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:59.667125 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:00:59.667132 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:00:59.667139 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:00:59.667406 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:00:59.667968 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:00:59.667980 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:00:59.668469 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 09:00:59.668821 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 09:00:59.669412 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":279,"threads_started":1}
I20260811 09:00:59.672457 10441 heartbeater.cc:347] Connected to a master server at 127.9.40.126:34091
I20260811 09:00:59.672859 10441 heartbeater.cc:464] Registering TS with master...
I20260811 09:00:59.673626 10441 heartbeater.cc:511] Master 127.9.40.126:34091 requested a full tablet report, sending...
I20260811 09:00:59.675233 10050 ts_manager.cc:194] Registered new tserver with Master: e3d37a8ee9bb401ab4155c0421e1b910 (127.9.40.66:43605)
I20260811 09:00:59.676095 10050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:39297
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:00:59.704452 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:00.031723 10447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:00.031999 10447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:00.032107 10447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:00.039222 10447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:00.039326 10447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:00.039438 10447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:00.048898 10447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:34091
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:00.050101 10447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:00.051384 10447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:00.064857 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:00.064915 10454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:00.065431 10453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:00.066191 10447 server_base.cc:1034] running on GCE node
I20260811 09:01:00.066814 10447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:00.067760 10447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:00.069015 10447 hybrid_clock.cc:648] HybridClock initialized: now 1786438860068954 us; error 36 us; skew 500 ppm
I20260811 09:01:00.069411 10447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:00.072921 10447 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:00.074491 10447 webserver.cc:460] Webserver started at http://127.9.40.67:34391/ using document root <none> and password file <none>
I20260811 09:01:00.075244 10447 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:00.075392 10447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:00.075762 10447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:00.079319 10447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "e2fd8570dfe6497d8016b9cac1905315"
format_stamp: "Formatted at 2026-08-11 09:01:00 on dist-test-slave-cbsf"
server_key: "93abc5d5d38cb6674aadd124b1a74b92"
server_key_iv: "69a56a39dd2030116e118d457b98535c"
server_key_version: "encryptionkey@0"
I20260811 09:01:00.080344 10447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "e2fd8570dfe6497d8016b9cac1905315"
format_stamp: "Formatted at 2026-08-11 09:01:00 on dist-test-slave-cbsf"
server_key: "93abc5d5d38cb6674aadd124b1a74b92"
server_key_iv: "69a56a39dd2030116e118d457b98535c"
server_key_version: "encryptionkey@0"
I20260811 09:01:00.086135 10447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:01:00.089776 10463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:00.091205 10447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:00.091385 10447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "e2fd8570dfe6497d8016b9cac1905315"
format_stamp: "Formatted at 2026-08-11 09:01:00 on dist-test-slave-cbsf"
server_key: "93abc5d5d38cb6674aadd124b1a74b92"
server_key_iv: "69a56a39dd2030116e118d457b98535c"
server_key_version: "encryptionkey@0"
I20260811 09:01:00.091596 10447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:00.107362 10447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:00.111155 10447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:00.111461 10447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:00.112416 10447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:00.113814 10447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:00.113888 10447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:00.114064 10447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:00.114116 10447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:00.148659 10447 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:46285
I20260811 09:01:00.148741 10595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:46285 every 8 connection(s)
I20260811 09:01:00.149999 10447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:00.153756 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10447
I20260811 09:01:00.153971 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:00.154554 9377 external_mini_cluster.cc:1428] Setting key b981effff9a69c4d6087fb0e9b8d61b8
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:00.171697 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.152600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43235 (local address 127.9.40.126:34091)
0811 09:01:00.152812 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.152817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.154214 (+ 1397us) server_negotiation.cc:407] Connection header received
0811 09:01:00.156835 (+ 2621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.156842 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.156942 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.157049 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.158746 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.159501 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.161164 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.161695 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.161787 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.165399 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.165403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.165406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.165437 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.168142 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.169213 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.169219 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.169223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.169362 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.169965 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.169970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.169973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.170364 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.170529 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.171272 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.171438 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:01:00.172895 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.153145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:34091 (local address 127.9.40.67:43235)
0811 09:01:00.154013 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.154062 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.156487 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.157260 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.157268 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.158038 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:00.158599 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.158616 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.159667 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.159671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.161028 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.161041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.161919 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.161925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.162143 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.162772 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:00.162794 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:00.165184 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:00.168297 (+ 3113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:00.168305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:00.168321 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:00.169076 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:00.169490 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:00.169495 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:00.169499 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:00.169830 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:00.170548 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:00.170557 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:00.170911 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.171156 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.171636 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":319,"threads_started":1}
I20260811 09:01:00.174310 10596 heartbeater.cc:347] Connected to a master server at 127.9.40.126:34091
I20260811 09:01:00.174654 10596 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:00.175416 10596 heartbeater.cc:511] Master 127.9.40.126:34091 requested a full tablet report, sending...
I20260811 09:01:00.176860 10050 ts_manager.cc:194] Registered new tserver with Master: e2fd8570dfe6497d8016b9cac1905315 (127.9.40.67:46285)
I20260811 09:01:00.177630 10050 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43235
I20260811 09:01:00.178818 10279 heartbeater.cc:503] Master 127.9.40.126:34091 was elected leader, sending a full tablet report...
I20260811 09:01:00.182669 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:01:00.203387 10281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.189495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44676 (local address 127.9.40.126:34091)
0811 09:01:00.189661 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.189666 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.190148 (+ 482us) server_negotiation.cc:407] Connection header received
0811 09:01:00.190272 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.190276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.190336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.190416 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.192069 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.192920 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.194188 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.194996 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.195160 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.197994 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.197999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.198004 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.198045 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.200907 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.201531 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.201535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.201537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.201662 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.202007 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.202010 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.202012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.202354 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.202541 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.202844 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.203042 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:01:00.209654 10050 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:00.216641 10050 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:00.238940 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.229249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.9.40.67:46285)
0811 09:01:00.231057 (+ 1808us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.231067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.231103 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:01:00.231243 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.231254 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.231541 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.232730 (+ 1189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:00.233032 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.234406 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.236191 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.237742 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.237873 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.238131 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.238439 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.238617 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1651,"thread_start_us":1371,"threads_started":1}
I20260811 09:01:00.239018 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.228927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46285 (local address 127.0.0.1:58442)
0811 09:01:00.229841 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.229879 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.230109 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.231856 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.231868 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.231915 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:00.232463 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.232480 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.234543 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.234550 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.236027 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.236039 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.237966 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.237972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.238191 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.238239 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.238381 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.238578 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":308,"threads_started":1}
I20260811 09:01:00.242054 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.231651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33771 (local address 127.0.0.1:54554)
0811 09:01:00.232269 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.232289 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.232412 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.233696 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.233703 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.233732 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:00.234270 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.234289 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.236601 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.236606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.239208 (+ 2602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.239222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.241263 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.241268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.241415 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.241444 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.241508 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.241615 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":60,"thread_start_us":640,"threads_started":1}
I20260811 09:01:00.242127 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.230333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43605 (local address 127.0.0.1:56250)
0811 09:01:00.231579 (+ 1246us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.231596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.231703 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.233902 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.233908 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.233959 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:00.234443 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.234454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.236602 (+ 2148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.236606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.238847 (+ 2241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.238861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.241171 (+ 2310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.241179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.241356 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.241404 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.241472 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.241607 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":54,"spinlock_wait_cycles":3456,"thread_start_us":976,"threads_started":1}
I20260811 09:01:00.242605 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.230889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56250 (local address 127.9.40.66:43605)
0811 09:01:00.233330 (+ 2441us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.233337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.233376 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:01:00.233463 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.233469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.233679 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.234644 (+ 965us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:00.234867 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.236427 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.239059 (+ 2632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.240899 (+ 1840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.241756 (+ 857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.241954 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.242043 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.242224 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2288,"thread_start_us":2795,"threads_started":1}
I20260811 09:01:00.243578 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.232144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.9.40.65:33771)
0811 09:01:00.233095 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.233106 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.233132 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:01:00.233223 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.233230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.233468 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.234521 (+ 1053us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:00.234772 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.236416 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.239437 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.241048 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.242697 (+ 1649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.242953 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.243023 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.243232 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":632,"threads_started":1}
I20260811 09:01:00.243352 10521 tablet_service.cc:1467] Processing CreateTablet for tablet 2174c300c4ba4499a6b2c1e71ef170b8 (DEFAULT_TABLE table=test-table [id=0fb9fff9fa464357b2e2b91337625c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:00.244997 10521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2174c300c4ba4499a6b2c1e71ef170b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:00.246371 10196 tablet_service.cc:1467] Processing CreateTablet for tablet 2174c300c4ba4499a6b2c1e71ef170b8 (DEFAULT_TABLE table=test-table [id=0fb9fff9fa464357b2e2b91337625c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:00.246305 10357 tablet_service.cc:1467] Processing CreateTablet for tablet 2174c300c4ba4499a6b2c1e71ef170b8 (DEFAULT_TABLE table=test-table [id=0fb9fff9fa464357b2e2b91337625c30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:00.247699 10196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2174c300c4ba4499a6b2c1e71ef170b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:00.247714 10357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2174c300c4ba4499a6b2c1e71ef170b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:00.259707 10616 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Bootstrap starting.
I20260811 09:01:00.261982 10617 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Bootstrap starting.
I20260811 09:01:00.263803 10618 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Bootstrap starting.
I20260811 09:01:00.266000 10616 tablet_bootstrap.cc:654] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:00.266458 10617 tablet_bootstrap.cc:654] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:00.266700 10618 tablet_bootstrap.cc:654] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:00.267381 10616 log.cc:824] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:00.267704 10618 log.cc:824] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:00.267688 10617 log.cc:824] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:00.270891 10617 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: No bootstrap required, opened a new log
I20260811 09:01:00.270892 10616 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: No bootstrap required, opened a new log
I20260811 09:01:00.271130 10618 tablet_bootstrap.cc:492] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: No bootstrap required, opened a new log
I20260811 09:01:00.271170 10617 ts_tablet_manager.cc:1397] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 09:01:00.271173 10616 ts_tablet_manager.cc:1397] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 09:01:00.271384 10618 ts_tablet_manager.cc:1397] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 09:01:00.278441 10618 raft_consensus.cc:348] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.278954 10618 raft_consensus.cc:374] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:00.279086 10618 raft_consensus.cc:729] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3d37a8ee9bb401ab4155c0421e1b910, State: Initialized, Role: FOLLOWER
I20260811 09:01:00.279747 10618 consensus_queue.cc:260] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.282159 10616 raft_consensus.cc:348] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.282155 10617 raft_consensus.cc:348] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.282574 10617 raft_consensus.cc:374] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:00.282662 10616 raft_consensus.cc:374] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:00.282688 10617 raft_consensus.cc:729] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2fd8570dfe6497d8016b9cac1905315, State: Initialized, Role: FOLLOWER
I20260811 09:01:00.282758 10441 heartbeater.cc:503] Master 127.9.40.126:34091 was elected leader, sending a full tablet report...
I20260811 09:01:00.282763 10616 raft_consensus.cc:729] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a7d31491284289831b6e0a7e0e45bf, State: Initialized, Role: FOLLOWER
I20260811 09:01:00.283349 10617 consensus_queue.cc:260] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.283320 10616 consensus_queue.cc:260] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.285686 10616 ts_tablet_manager.cc:1428] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 09:01:00.287124 10618 ts_tablet_manager.cc:1428] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
I20260811 09:01:00.288123 10596 heartbeater.cc:503] Master 127.9.40.126:34091 was elected leader, sending a full tablet report...
I20260811 09:01:00.288817 10617 ts_tablet_manager.cc:1428] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
I20260811 09:01:00.318288 10622 raft_consensus.cc:482] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:00.318622 10622 raft_consensus.cc:504] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.320745 10622 leader_election.cc:288] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e2fd8570dfe6497d8016b9cac1905315 (127.9.40.67:46285), a0a7d31491284289831b6e0a7e0e45bf (127.9.40.65:33771)
I20260811 09:01:00.328440 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.322542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39225 (local address 127.9.40.65:33771)
0811 09:01:00.322777 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.322784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.322858 (+ 74us) server_negotiation.cc:407] Connection header received
0811 09:01:00.323059 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.323065 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.323154 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.323297 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:00.323961 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.324693 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.326671 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.327497 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.327668 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.327782 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.328067 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.328190 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:01:00.328629 10627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.322103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33771 (local address 127.9.40.66:39225)
0811 09:01:00.322780 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.322793 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.322897 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.323319 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.323322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.323353 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:00.323789 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.323796 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.324882 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.324887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.326498 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.326510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.327729 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.327736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.327888 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.327912 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.328031 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.328160 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":25,"thread_start_us":372,"threads_started":1}
I20260811 09:01:00.329152 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.321354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:41439 (local address 127.9.40.67:46285)
0811 09:01:00.321633 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.321643 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.322237 (+ 594us) server_negotiation.cc:407] Connection header received
0811 09:01:00.322602 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.322609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.322702 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.322786 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:00.323950 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.324932 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.326992 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.327947 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.328147 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.328291 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.328765 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.328908 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:01:00.329787 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.321246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46285 (local address 127.9.40.66:41439)
0811 09:01:00.322067 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:00.322122 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:00.322348 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:00.323090 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:00.323095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:00.323126 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:00.323754 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.323775 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.325092 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.325099 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.326807 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:00.326822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.328234 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.328242 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:00.328452 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.328480 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:01:00.328723 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 09:01:00.328867 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":2,"spinlock_wait_cycles":27008,"thread_start_us":410,"threads_started":1}
I20260811 09:01:00.330097 10221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2174c300c4ba4499a6b2c1e71ef170b8" candidate_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a7d31491284289831b6e0a7e0e45bf" is_pre_election: true
I20260811 09:01:00.331003 10221 raft_consensus.cc:2454] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d37a8ee9bb401ab4155c0421e1b910 in term 0.
I20260811 09:01:00.330993 10543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2174c300c4ba4499a6b2c1e71ef170b8" candidate_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2fd8570dfe6497d8016b9cac1905315" is_pre_election: true
I20260811 09:01:00.331682 10543 raft_consensus.cc:2454] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3d37a8ee9bb401ab4155c0421e1b910 in term 0.
I20260811 09:01:00.331866 10305 leader_election.cc:302] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a7d31491284289831b6e0a7e0e45bf, e3d37a8ee9bb401ab4155c0421e1b910; no voters:
I20260811 09:01:00.332356 10622 raft_consensus.cc:2781] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:00.332543 10622 raft_consensus.cc:482] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:00.332684 10622 raft_consensus.cc:3037] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:00.335359 10622 raft_consensus.cc:504] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.336113 10622 leader_election.cc:288] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [CANDIDATE]: Term 1 election: Requested vote from peers e2fd8570dfe6497d8016b9cac1905315 (127.9.40.67:46285), a0a7d31491284289831b6e0a7e0e45bf (127.9.40.65:33771)
I20260811 09:01:00.336567 10543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2174c300c4ba4499a6b2c1e71ef170b8" candidate_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2fd8570dfe6497d8016b9cac1905315"
I20260811 09:01:00.336580 10221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2174c300c4ba4499a6b2c1e71ef170b8" candidate_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a7d31491284289831b6e0a7e0e45bf"
I20260811 09:01:00.336805 10221 raft_consensus.cc:3037] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:00.336838 10543 raft_consensus.cc:3037] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:00.339076 10221 raft_consensus.cc:2454] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d37a8ee9bb401ab4155c0421e1b910 in term 1.
I20260811 09:01:00.339483 10305 leader_election.cc:302] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a7d31491284289831b6e0a7e0e45bf, e3d37a8ee9bb401ab4155c0421e1b910; no voters:
I20260811 09:01:00.339572 10543 raft_consensus.cc:2454] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3d37a8ee9bb401ab4155c0421e1b910 in term 1.
I20260811 09:01:00.339745 10622 raft_consensus.cc:2781] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:00.340283 10622 raft_consensus.cc:686] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [term 1 LEADER]: Becoming Leader. State: Replica: e3d37a8ee9bb401ab4155c0421e1b910, State: Running, Role: LEADER
I20260811 09:01:00.340844 10622 consensus_queue.cc:237] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } }
I20260811 09:01:00.346243 10048 catalog_manager.cc:5495] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3d37a8ee9bb401ab4155c0421e1b910 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2fd8570dfe6497d8016b9cac1905315" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3d37a8ee9bb401ab4155c0421e1b910" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43605 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0a7d31491284289831b6e0a7e0e45bf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33771 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:00.371343 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.356875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.9.40.65:33771)
0811 09:01:00.357116 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.357126 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.357563 (+ 437us) server_negotiation.cc:407] Connection header received
0811 09:01:00.357701 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.357705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.357769 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.357852 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.359372 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.360116 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.361434 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.361944 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.362044 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.366025 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.366039 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.366063 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.366098 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.368875 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.369489 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.369496 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.369498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.369621 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.369935 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.369938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.369940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.370318 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.370543 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.370806 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.370964 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 09:01:00.395550 10280 tablet.cc:2369] T 2174c300c4ba4499a6b2c1e71ef170b8 P a0a7d31491284289831b6e0a7e0e45bf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:00.399994 10442 tablet.cc:2369] T 2174c300c4ba4499a6b2c1e71ef170b8 P e3d37a8ee9bb401ab4155c0421e1b910: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:00.401670 10597 tablet.cc:2369] T 2174c300c4ba4499a6b2c1e71ef170b8 P e2fd8570dfe6497d8016b9cac1905315: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:00.426512 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.409533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.9.40.65:33771)
0811 09:01:00.409842 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.409852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.410139 (+ 287us) server_negotiation.cc:407] Connection header received
0811 09:01:00.410454 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.410459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.410541 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.410677 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.412372 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.413362 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.414921 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.415418 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.415579 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.420751 (+ 5172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.420763 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.420775 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.420818 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.423580 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.424221 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.424226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.424230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.424370 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.424793 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.424797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.424799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.425133 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.425554 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.425816 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.426115 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:00.479059 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.463099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58028 (local address 127.9.40.65:33771)
0811 09:01:00.463397 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.463406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.463710 (+ 304us) server_negotiation.cc:407] Connection header received
0811 09:01:00.463958 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.463963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.464034 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.464151 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.466246 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.467011 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.468512 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.469185 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.469328 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.473543 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.473549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.473552 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.473602 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.476226 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.476984 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.476990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.477000 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.477121 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.477548 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.477552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.477554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.477846 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.478127 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.478442 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.478702 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 09:01:00.481484 10174 tablet_service.cc:2908] Rejecting scan request for tablet 2174c300c4ba4499a6b2c1e71ef170b8: Uninitialized: clean time has not yet been initialized
I20260811 09:01:00.484944 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10123
I20260811 09:01:00.498636 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.486114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.9.40.65:33771)
0811 09:01:00.486386 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.486394 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.486642 (+ 248us) server_negotiation.cc:407] Connection header received
0811 09:01:00.486840 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.486845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.486935 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.487041 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.488308 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.489079 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.490291 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.490798 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.490941 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.493211 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.493224 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.493232 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.493261 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.495675 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.496375 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.496383 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.496388 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.496571 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.497019 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.497026 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.497029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.497403 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.497720 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.498043 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.498264 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260811 09:01:00.499571 10271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:00.596483 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10123
I20260811 09:01:00.618268 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10286
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:00.632324 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.619091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.9.40.66:43605)
0811 09:01:00.619273 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.619278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.619296 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:01:00.619409 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.619414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.619507 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.619594 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.620787 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.621538 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.622827 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.623196 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.623302 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.627156 (+ 3854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.627168 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.627185 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.627229 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.629926 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.630540 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.630548 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.630550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.630695 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.631002 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.631005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.631007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.631364 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.631623 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.631790 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.631961 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:01:00.633208 10436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:00.723345 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10286
I20260811 09:01:00.743592 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10447
Aug 11 09:01:00 dist-test-slave-cbsf krb5kdc[9996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:00.757958 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:00.744380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43816 (local address 127.9.40.67:46285)
0811 09:01:00.744572 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:00.744577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:00.744596 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:00.744785 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:00.744789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:00.744861 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:00.744944 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:00.746173 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.746903 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.748088 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:00.748474 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:00.748566 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:00.752579 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:00.752594 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:00.752611 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:00.752647 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:00.755203 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.755862 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.755869 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.755871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.755993 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:00.756396 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:00.756400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:00.756402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:00.756936 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:00.757258 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:00.757431 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:01:00.757601 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:00.758680 10589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:00.845803 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10447
I20260811 09:01:00.866302 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10012
I20260811 09:01:00.867443 10082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:00.975107 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10012
2026-08-11T09:01:00Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4300 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:01 dist-test-slave-cbsf krb5kdc[10648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:01 dist-test-slave-cbsf krb5kdc[10648](info): set up 2 sockets
Aug 11 09:01:01 dist-test-slave-cbsf krb5kdc[10648](info): commencing operation
krb5kdc: starting...
W20260811 09:01:02.144831 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:02 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:02.346308 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33103
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:02.671967 10662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:02.672243 10662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:02.672392 10662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:02.679494 10662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:02.679615 10662 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:01:02.679705 10662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:02.679800 10662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:02.679870 10662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:02.679953 10662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:02.689666 10662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33103
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:02.690944 10662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:02.692287 10662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:02.704298 10670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:02.704452 10671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:02.704465 10673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:02.705605 10662 server_base.cc:1034] running on GCE node
I20260811 09:01:02.706355 10662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:02.707688 10662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:02.708957 10662 hybrid_clock.cc:648] HybridClock initialized: now 1786438862708897 us; error 46 us; skew 500 ppm
I20260811 09:01:02.709297 10662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:02 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438862, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:02.712337 10662 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:02.713706 10662 webserver.cc:460] Webserver started at http://127.9.40.126:33799/ using document root <none> and password file <none>
I20260811 09:01:02.714334 10662 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:02.714413 10662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:02.714669 10662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:02.717618 10662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "bae197e7ec5a49e2b8b17d6ab5028c93"
format_stamp: "Formatted at 2026-08-11 09:01:02 on dist-test-slave-cbsf"
server_key: "2cf8f4aa687bf67bb888e517d39328f4"
server_key_iv: "ca3ac5fe01a8f83209894e0d201ce5d9"
server_key_version: "encryptionkey@0"
I20260811 09:01:02.718377 10662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bae197e7ec5a49e2b8b17d6ab5028c93"
format_stamp: "Formatted at 2026-08-11 09:01:02 on dist-test-slave-cbsf"
server_key: "2cf8f4aa687bf67bb888e517d39328f4"
server_key_iv: "ca3ac5fe01a8f83209894e0d201ce5d9"
server_key_version: "encryptionkey@0"
I20260811 09:01:02.723163 10662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:01:02.726482 10682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:02.727809 10662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:01:02.727959 10662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "bae197e7ec5a49e2b8b17d6ab5028c93"
format_stamp: "Formatted at 2026-08-11 09:01:02 on dist-test-slave-cbsf"
server_key: "2cf8f4aa687bf67bb888e517d39328f4"
server_key_iv: "ca3ac5fe01a8f83209894e0d201ce5d9"
server_key_version: "encryptionkey@0"
I20260811 09:01:02.728132 10662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:02.744410 10662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:02.748319 10662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:02.748529 10662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:02.770659 10662 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33103
I20260811 09:01:02.770666 10750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33103 every 8 connection(s)
I20260811 09:01:02.772061 10662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:02.774662 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10662
I20260811 09:01:02.774926 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:02.775614 9377 external_mini_cluster.cc:1428] Setting key 06d2de804251dc5192a2cf3df9b902de
I20260811 09:01:02.776059 10751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:02.790182 10751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Bootstrap starting.
Aug 11 09:01:02 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:02.793957 10751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:02.795073 10751 log.cc:824] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:02.797873 10751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: No bootstrap required, opened a new log
I20260811 09:01:02.799686 10754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:02.777738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.9.40.126:33103)
0811 09:01:02.778526 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:02.778541 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:02.778607 (+ 66us) server_negotiation.cc:407] Connection header received
0811 09:01:02.780925 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:02.780955 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:02.781460 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:02.781887 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:02.783271 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:02.785196 (+ 1925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:02.786384 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:02.787554 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:02.787743 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:02.791905 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:02.791915 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:02.791934 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:02.791988 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:02.795632 (+ 3644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:02.796313 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:02.796325 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:02.796331 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:02.796506 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:02.796939 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:02.796944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:02.796947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:02.797513 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:02.797825 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:02.798292 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 09:01:02.798722 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":309,"threads_started":1}
I20260811 09:01:02.805855 10751 raft_consensus.cc:348] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } }
I20260811 09:01:02.806319 10751 raft_consensus.cc:374] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:02.806429 10751 raft_consensus.cc:729] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bae197e7ec5a49e2b8b17d6ab5028c93, State: Initialized, Role: FOLLOWER
I20260811 09:01:02.806967 10751 consensus_queue.cc:260] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } }
I20260811 09:01:02.807217 10751 raft_consensus.cc:388] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:02.807365 10751 raft_consensus.cc:482] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:02.807536 10751 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:02.809854 10751 raft_consensus.cc:504] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } }
I20260811 09:01:02.810359 10751 leader_election.cc:302] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bae197e7ec5a49e2b8b17d6ab5028c93; no voters:
I20260811 09:01:02.810957 10751 leader_election.cc:288] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:02.811120 10756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:02.812003 10756 raft_consensus.cc:686] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [term 1 LEADER]: Becoming Leader. State: Replica: bae197e7ec5a49e2b8b17d6ab5028c93, State: Running, Role: LEADER
I20260811 09:01:02.812597 10756 consensus_queue.cc:237] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } }
I20260811 09:01:02.813221 10751 sys_catalog.cc:564] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:02.815750 10759 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bae197e7ec5a49e2b8b17d6ab5028c93. Latest consensus state: current_term: 1 leader_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } } }
I20260811 09:01:02.815837 10758 sys_catalog.cc:455] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bae197e7ec5a49e2b8b17d6ab5028c93" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33103 } } }
I20260811 09:01:02.816123 10759 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:02.817145 10758 sys_catalog.cc:458] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:02.818228 10764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:02.824779 10764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:02.833479 10764 catalog_manager.cc:1347] Generated new cluster ID: bf856ae1aed24253a34b415894fb0b32
I20260811 09:01:02.833596 10764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:02.858760 10764 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:02.860486 10764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:02.871407 10764 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Generated new TSK 0
I20260811 09:01:02.872311 10764 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:02.928478 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33103
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:03.282974 10780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:03.283264 10780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:03.283365 10780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:03.290463 10780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:03.290577 10780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:03.290661 10780 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:03.290743 10780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:03.300755 10780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:03.302146 10780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:03.303472 10780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:03.316793 10785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:03.316669 10786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:03.316931 10788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:03.317821 10780 server_base.cc:1034] running on GCE node
I20260811 09:01:03.318512 10780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:03.319469 10780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:03.320771 10780 hybrid_clock.cc:648] HybridClock initialized: now 1786438863320653 us; error 60 us; skew 500 ppm
I20260811 09:01:03.321077 10780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:03 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438863, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:03.328923 10780 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:03.330444 10780 webserver.cc:460] Webserver started at http://127.9.40.65:33609/ using document root <none> and password file <none>
I20260811 09:01:03.331162 10780 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:03.331298 10780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:03.331700 10780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:03.334658 10780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "73c07d17aca8431c98e83e39d573d9fd"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "9d4ca5a75554fbce7c2f0a8568d31877"
server_key_iv: "9123539910f38e261fc8183bc783c2e2"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.335441 10780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "73c07d17aca8431c98e83e39d573d9fd"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "9d4ca5a75554fbce7c2f0a8568d31877"
server_key_iv: "9123539910f38e261fc8183bc783c2e2"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.340497 10780 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:01:03.343779 10795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:03.345191 10780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:03.345383 10780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "73c07d17aca8431c98e83e39d573d9fd"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "9d4ca5a75554fbce7c2f0a8568d31877"
server_key_iv: "9123539910f38e261fc8183bc783c2e2"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.345604 10780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:03.380613 10780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:03.385289 10780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:03.385615 10780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:03.388520 10780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:03.388630 10780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:03.388779 10780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:03.388860 10780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:03 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438863, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:03.414336 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.389393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33631 (local address 127.9.40.126:33103)
0811 09:01:03.390029 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:03.390035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:03.391092 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 09:01:03.394100 (+ 3008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:03.394106 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:03.394188 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:03.394336 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:03.396889 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.397996 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.400486 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.401118 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.401256 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.406131 (+ 4875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:03.406141 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:03.406149 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:03.406207 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:03.410240 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.411002 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.411011 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.411018 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.411183 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.411755 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.411760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.411763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.412106 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:03.412271 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:03.413577 (+ 1306us) server_negotiation.cc:299] Negotiation successful
0811 09:01:03.413860 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":391,"threads_started":1}
I20260811 09:01:03.416123 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.390008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.65:33631)
0811 09:01:03.390897 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:03.390948 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:03.393755 (+ 2807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:03.394794 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:03.394810 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:03.395671 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:03.396584 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.396617 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.398196 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.398200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.400332 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.400350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.401400 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.401408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.401653 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.402518 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:03.402556 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:03.405859 (+ 3303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:03.410395 (+ 4536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.410406 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.410428 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.410864 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.411311 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.411318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.411324 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.411625 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.412269 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:03.412283 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:03.412990 (+ 707us) client_negotiation.cc:769] Sending connection context
0811 09:01:03.413377 (+ 387us) client_negotiation.cc:240] Negotiation successful
0811 09:01:03.414118 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":346,"threads_started":1}
I20260811 09:01:03.436952 10780 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:44919
I20260811 09:01:03.437008 10932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:44919 every 8 connection(s)
I20260811 09:01:03.438550 10780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:03.440810 10933 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33103
I20260811 09:01:03.441058 10933 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:03.441699 10933 heartbeater.cc:511] Master 127.9.40.126:33103 requested a full tablet report, sending...
I20260811 09:01:03.443053 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10780
I20260811 09:01:03.443284 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:03.443948 10700 ts_manager.cc:194] Registered new tserver with Master: 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919)
I20260811 09:01:03.444057 9377 external_mini_cluster.cc:1428] Setting key b7668f8d7f7ed1e4560520af42f9325d
I20260811 09:01:03.446069 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:33631
I20260811 09:01:03.446664 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.424266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:37003 (local address 127.9.40.126:33103)
0811 09:01:03.424460 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:03.424467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:03.431399 (+ 6932us) server_negotiation.cc:407] Connection header received
0811 09:01:03.431592 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:03.431596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:03.431663 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:03.431797 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:03.433663 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.434626 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.436836 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.437454 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.437553 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.440154 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:03.440164 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:03.440171 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:03.440215 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:03.443534 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.444341 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.444346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.444350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.444519 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.444987 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.444997 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.445003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.445422 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:03.445620 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:03.446042 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 09:01:03.446298 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:01:03.446673 10899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.424312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.65:37003)
0811 09:01:03.431265 (+ 6953us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:03.431286 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:03.431468 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:03.431864 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:03.431872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:03.432866 (+ 994us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:03.433467 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.433487 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.434804 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.434809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.436634 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.436659 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.437649 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.437656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.437837 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.438488 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:03.438523 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:03.439852 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:03.443673 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.443682 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.443687 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.444193 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.444647 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.444652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.444656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.444848 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.445551 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:03.445557 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:03.445736 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:01:03.445888 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 09:01:03.446209 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6815,"thread_start_us":6550,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:03.490495 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33103
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:03.814281 10938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:03.814657 10938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:03.814837 10938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:03.822445 10938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:03.822556 10938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:03.822722 10938 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:03.822839 10938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:03.832926 10938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:03.834205 10938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:03.835618 10938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:03.849045 10943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:03.849080 10945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:03.849251 10947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:03.850322 10938 server_base.cc:1034] running on GCE node
I20260811 09:01:03.850941 10938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:03.851815 10938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:03.853052 10938 hybrid_clock.cc:648] HybridClock initialized: now 1786438863852989 us; error 58 us; skew 500 ppm
I20260811 09:01:03.853341 10938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:03 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438863, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:03.856549 10938 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:03.858067 10938 webserver.cc:460] Webserver started at http://127.9.40.66:46243/ using document root <none> and password file <none>
I20260811 09:01:03.858757 10938 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:03.858891 10938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:03.859237 10938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:03.862532 10938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "d4bb8e6756134d71af8b6f84299c9876"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "456b1dddd6034bf3b143dc84beba9396"
server_key_iv: "0aab7b30734354000db5b6ca033c9313"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.863356 10938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "d4bb8e6756134d71af8b6f84299c9876"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "456b1dddd6034bf3b143dc84beba9396"
server_key_iv: "0aab7b30734354000db5b6ca033c9313"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.868449 10938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:01:03.872140 10955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:03.873524 10938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:03.873654 10938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "d4bb8e6756134d71af8b6f84299c9876"
format_stamp: "Formatted at 2026-08-11 09:01:03 on dist-test-slave-cbsf"
server_key: "456b1dddd6034bf3b143dc84beba9396"
server_key_iv: "0aab7b30734354000db5b6ca033c9313"
server_key_version: "encryptionkey@0"
I20260811 09:01:03.873826 10938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:03.898480 10938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:03.902873 10938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:03.903095 10938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:03.905786 10938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:03.905911 10938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:03.906066 10938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:03.906159 10938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:03 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438863, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:03.929265 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.906222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39075 (local address 127.9.40.126:33103)
0811 09:01:03.906468 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:03.906477 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:03.907928 (+ 1451us) server_negotiation.cc:407] Connection header received
0811 09:01:03.910043 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:03.910052 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:03.910194 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:03.910306 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:03.912757 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.914241 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.917610 (+ 3369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.918308 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.918444 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.922928 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:03.922936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:03.922946 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:03.922980 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:03.926117 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.926917 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.926922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.926927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.927049 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.927477 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.927482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.927486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.927784 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:03.927947 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:03.928652 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 09:01:03.928899 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:01:03.930984 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.906834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.66:39075)
0811 09:01:03.907728 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:03.907785 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:03.909764 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:03.910589 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:03.910606 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:03.911368 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:03.912476 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.912498 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.914459 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.914464 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.917422 (+ 2958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.917442 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.918875 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.918887 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.919225 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.920000 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:03.920022 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:03.922711 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:03.926267 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.926276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.926291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.926785 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.927163 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.927167 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.927171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.927356 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.927915 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:03.927924 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:03.928270 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 09:01:03.928520 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 09:01:03.929094 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":371,"threads_started":1}
I20260811 09:01:03.952944 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.937192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:54275 (local address 127.9.40.126:33103)
0811 09:01:03.937380 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:03.937388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:03.938319 (+ 931us) server_negotiation.cc:407] Connection header received
0811 09:01:03.938503 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:03.938507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:03.938574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:03.938651 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:03.940129 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.940959 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.943438 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.943814 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.943902 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.946600 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:03.946605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:03.946608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:03.946639 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:03.950015 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.950803 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.950810 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.950814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.950973 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:03.951470 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:03.951473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:03.951476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:03.951820 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:03.951934 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:03.952482 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 09:01:03.952653 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:01:03.953545 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:03.937330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.66:54275)
0811 09:01:03.938191 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:03.938215 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:03.938384 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:03.938728 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:03.938736 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:03.939492 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:03.939969 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.939985 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.941116 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.941124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.943287 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:03.943307 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:03.944001 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:03.944008 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:03.944254 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:03.945150 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:03.945185 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:03.946362 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:03.950157 (+ 3795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.950161 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.950165 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.950687 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.951112 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:03.951120 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:03.951127 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:03.951337 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:03.951950 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:03.951957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:03.952218 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 09:01:03.952409 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 09:01:03.952905 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":460,"threads_started":1}
I20260811 09:01:03.968153 10938 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45079
I20260811 09:01:03.968195 11091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45079 every 8 connection(s)
I20260811 09:01:03.969444 10938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:03.971551 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 10938
I20260811 09:01:03.971792 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:03.971872 11092 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33103
I20260811 09:01:03.972112 11092 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:03.972337 9377 external_mini_cluster.cc:1428] Setting key 6f4137f7fc2961d99b69f6ae9490b9bc
I20260811 09:01:03.972857 11092 heartbeater.cc:511] Master 127.9.40.126:33103 requested a full tablet report, sending...
I20260811 09:01:03.974251 10700 ts_manager.cc:194] Registered new tserver with Master: d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079)
I20260811 09:01:03.975371 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:39075
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:04.016331 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33103
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:04.340651 11096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:04.341027 11096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:04.341204 11096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:04.348384 11096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:04.348533 11096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:04.348730 11096 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:04.348879 11096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:04.358915 11096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33103
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:04.360230 11096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:04.361650 11096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:04.376456 11102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:04.376502 11103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:04.376544 11108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:04.377596 11096 server_base.cc:1034] running on GCE node
I20260811 09:01:04.378197 11096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:04.379134 11096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:04.380379 11096 hybrid_clock.cc:648] HybridClock initialized: now 1786438864380321 us; error 32 us; skew 500 ppm
I20260811 09:01:04.380682 11096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:04 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:04.383806 11096 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:04.385330 11096 webserver.cc:460] Webserver started at http://127.9.40.67:40225/ using document root <none> and password file <none>
I20260811 09:01:04.385970 11096 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:04.386059 11096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:04.386327 11096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:04.389225 11096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc"
format_stamp: "Formatted at 2026-08-11 09:01:04 on dist-test-slave-cbsf"
server_key: "587b249cee57445235b6efa2c669ee7d"
server_key_iv: "80e6d8568607b35e648b27b92332cb08"
server_key_version: "encryptionkey@0"
I20260811 09:01:04.389993 11096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc"
format_stamp: "Formatted at 2026-08-11 09:01:04 on dist-test-slave-cbsf"
server_key: "587b249cee57445235b6efa2c669ee7d"
server_key_iv: "80e6d8568607b35e648b27b92332cb08"
server_key_version: "encryptionkey@0"
I20260811 09:01:04.395164 11096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:01:04.398804 11115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:04.400341 11096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:04.400525 11096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc"
format_stamp: "Formatted at 2026-08-11 09:01:04 on dist-test-slave-cbsf"
server_key: "587b249cee57445235b6efa2c669ee7d"
server_key_iv: "80e6d8568607b35e648b27b92332cb08"
server_key_version: "encryptionkey@0"
I20260811 09:01:04.400799 11096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:04.425719 11096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:04.430461 11096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:04.430761 11096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:04.433858 11096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:04.433974 11096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:04.434146 11096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:04.434237 11096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:04 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:04.455399 10933 heartbeater.cc:503] Master 127.9.40.126:33103 was elected leader, sending a full tablet report...
I20260811 09:01:04.460256 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.434647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41533 (local address 127.9.40.126:33103)
0811 09:01:04.434840 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.434846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.436294 (+ 1448us) server_negotiation.cc:407] Connection header received
0811 09:01:04.439264 (+ 2970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.439271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.439376 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.439450 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:04.441809 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.443289 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.445883 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.446656 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.446824 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.451656 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:04.451660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:04.451664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:04.451697 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:04.455944 (+ 4247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.457173 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.457181 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.457187 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.457400 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.457896 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.457902 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.457905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.458212 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:04.458379 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.459361 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.459716 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:01:04.460671 11125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.435210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.67:41533)
0811 09:01:04.436099 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.436164 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.438899 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.440002 (+ 1103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.440014 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.440822 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:04.441583 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.441605 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.443491 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.443498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.445683 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.445702 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.446960 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.446969 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.447208 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.448036 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:04.448056 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:04.451423 (+ 3367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:04.456133 (+ 4710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:04.456145 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:04.456166 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:04.456954 (+ 788us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:04.457535 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:04.457539 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:04.457542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:04.457760 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:04.458343 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:04.458353 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:04.458852 (+ 499us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.459187 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.459655 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":400,"threads_started":1}
I20260811 09:01:04.483600 11096 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:40119
I20260811 09:01:04.483659 11250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:40119 every 8 connection(s)
I20260811 09:01:04.484544 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.467377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.9.40.67:43131)
0811 09:01:04.469504 (+ 2127us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.469538 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.469684 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.470237 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.470241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.471054 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:04.471490 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.471500 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.472973 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.472984 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.474433 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.474449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.475516 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.475523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.475708 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.476433 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:04.476450 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:04.477825 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:04.481803 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:04.481810 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:04.481814 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:04.482225 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:04.482610 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:04.482616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:04.482619 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:04.482825 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:04.483558 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:04.483565 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:04.483804 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.483964 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.484224 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1899,"thread_start_us":1710,"threads_started":1}
I20260811 09:01:04.484686 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.467229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43131 (local address 127.9.40.126:33103)
0811 09:01:04.467412 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.467420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.469617 (+ 2197us) server_negotiation.cc:407] Connection header received
0811 09:01:04.469812 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.469817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.469900 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.470012 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:04.471721 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.472665 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.474587 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.475309 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.475436 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.478035 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:04.478043 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:04.478050 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:04.478109 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:04.481675 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.482354 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.482360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.482363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.482492 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.482970 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.482978 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.482984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.483398 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:04.483575 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.484093 (+ 518us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.484323 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:01:04.485260 11096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:04.485545 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 11096
I20260811 09:01:04.485755 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:04.486210 9377 external_mini_cluster.cc:1428] Setting key 72510eb6c47d6e781f9cc588ec43c457
I20260811 09:01:04.489180 11251 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33103
I20260811 09:01:04.489451 11251 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:04.490370 11251 heartbeater.cc:511] Master 127.9.40.126:33103 requested a full tablet report, sending...
I20260811 09:01:04.491734 10700 ts_manager.cc:194] Registered new tserver with Master: 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119)
I20260811 09:01:04.492488 10700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:41533
I20260811 09:01:04.503553 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:01:04 dist-test-slave-cbsf krb5kdc[10648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:01:04 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:04.556823 10806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.542756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36724 (local address 127.9.40.126:33103)
0811 09:01:04.543012 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.543020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.543201 (+ 181us) server_negotiation.cc:407] Connection header received
0811 09:01:04.543425 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.543430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.543496 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.543606 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:04.545095 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.545819 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.547135 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.547580 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.547716 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.551664 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:04.551669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:04.551672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:04.551705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:04.554217 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.554829 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.554836 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.554840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.554977 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:04.555365 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:04.555370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:04.555375 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:04.555641 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:04.555902 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.556184 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.556418 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:01:04.561129 10700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:04.568387 10700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:04.595821 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.581821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.9.40.65:44919)
0811 09:01:04.585418 (+ 3597us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.585424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.585448 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:01:04.585561 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.585569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.585815 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.586991 (+ 1176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:04.587244 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.588862 (+ 1618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.592334 (+ 3472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.594299 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.594588 (+ 289us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.594878 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.595266 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.595498 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3447,"thread_start_us":3275,"threads_started":1}
I20260811 09:01:04.596060 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.581505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:44919 (local address 127.0.0.1:37784)
0811 09:01:04.582629 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.582680 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.582856 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.586107 (+ 3251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.586114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.586337 (+ 223us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:04.586772 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.586785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.589049 (+ 2264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.589072 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.592094 (+ 3022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.592108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.594594 (+ 2486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.594605 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.594940 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.595021 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.595197 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.595457 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"mutex_wait_us":31,"thread_start_us":1239,"threads_started":1}
I20260811 09:01:04.597831 11267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.582323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45079 (local address 127.0.0.1:58020)
0811 09:01:04.585715 (+ 3392us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.585735 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.585856 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.586881 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.586889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.586937 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:04.587459 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.587474 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.589491 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.589498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.592342 (+ 2844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.592358 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.596659 (+ 4301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.596668 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.596946 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.596996 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.597094 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.597321 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3265,"mutex_wait_us":1,"thread_start_us":3173,"threads_started":1}
I20260811 09:01:04.598876 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.582977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.9.40.66:45079)
0811 09:01:04.583607 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.583616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.585997 (+ 2381us) server_negotiation.cc:407] Connection header received
0811 09:01:04.586110 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.586120 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.586390 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.586610 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:04.587710 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.589317 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.592544 (+ 3227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.594050 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.594192 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.594407 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.598222 (+ 3815us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.598474 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":292,"threads_started":1}
I20260811 09:01:04.601508 10855 tablet_service.cc:1467] Processing CreateTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c (DEFAULT_TABLE table=test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.603271 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c2910df2d64615a0aaefaebd1cb51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.607245 11011 tablet_service.cc:1467] Processing CreateTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c (DEFAULT_TABLE table=test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.608759 11011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c2910df2d64615a0aaefaebd1cb51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.612201 11264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.585844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40119 (local address 127.0.0.1:60616)
0811 09:01:04.596364 (+ 10520us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.596389 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.596508 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.605836 (+ 9328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.605841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.605867 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:04.606059 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.606070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.607443 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.607449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.609381 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.609401 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.611289 (+ 1888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.611297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.611499 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.611545 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.611627 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.611814 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10389,"thread_start_us":17886,"threads_started":1}
I20260811 09:01:04.613517 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.586139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60616 (local address 127.9.40.67:40119)
0811 09:01:04.605364 (+ 19225us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.605370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.605397 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:01:04.605473 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.605478 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.605645 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.605865 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:04.606202 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.607280 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.609606 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.611095 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.612475 (+ 1380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.612796 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.612880 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.613089 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19080,"thread_start_us":18868,"threads_started":1}
I20260811 09:01:04.617697 11170 tablet_service.cc:1467] Processing CreateTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c (DEFAULT_TABLE table=test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.619755 11170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89c2910df2d64615a0aaefaebd1cb51c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.621918 11272 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Bootstrap starting.
I20260811 09:01:04.625805 11272 tablet_bootstrap.cc:654] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.627111 11272 log.cc:824] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:04.631279 11272 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: No bootstrap required, opened a new log
I20260811 09:01:04.631757 11272 ts_tablet_manager.cc:1397] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 09:01:04.635010 11274 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Bootstrap starting.
I20260811 09:01:04.638525 11274 tablet_bootstrap.cc:654] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.640933 11274 log.cc:824] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:04.648082 11275 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Bootstrap starting.
I20260811 09:01:04.647955 11272 raft_consensus.cc:348] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.648665 11272 raft_consensus.cc:374] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.648926 11272 raft_consensus.cc:729] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c07d17aca8431c98e83e39d573d9fd, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.650843 11272 consensus_queue.cc:260] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.652014 11274 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: No bootstrap required, opened a new log
I20260811 09:01:04.652374 11275 tablet_bootstrap.cc:654] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.653780 11274 ts_tablet_manager.cc:1397] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260811 09:01:04.653815 11275 log.cc:824] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:04.656703 11277 raft_consensus.cc:482] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:04.657081 11277 raft_consensus.cc:504] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.659667 11272 ts_tablet_manager.cc:1428] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Time spent starting tablet: real 0.028s user 0.012s sys 0.011s
I20260811 09:01:04.660427 11277 leader_election.cc:288] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079), 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119)
I20260811 09:01:04.661005 11275 tablet_bootstrap.cc:492] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: No bootstrap required, opened a new log
I20260811 09:01:04.661286 11275 ts_tablet_manager.cc:1397] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260811 09:01:04.666496 11274 raft_consensus.cc:348] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.666946 11274 raft_consensus.cc:374] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.667064 11274 raft_consensus.cc:729] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fafbc9fe1034cb8a9dba365b5eea8cc, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.667776 11274 consensus_queue.cc:260] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.673204 11269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.661190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33345 (local address 127.9.40.66:45079)
0811 09:01:04.661452 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.661462 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.662354 (+ 892us) server_negotiation.cc:407] Connection header received
0811 09:01:04.662662 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.662672 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.662771 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.662848 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:04.663854 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.665488 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.669436 (+ 3948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.671443 (+ 2007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.671599 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.671767 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.672573 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.672786 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:01:04.674417 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.660609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45079 (local address 127.9.40.65:33345)
0811 09:01:04.662259 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.662280 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.662469 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.663115 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.663123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.663159 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:04.663636 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.663653 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.665719 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.665728 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.669202 (+ 3474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.669225 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.672041 (+ 2816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.672050 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.672328 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.672383 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.673681 (+ 1298us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.673889 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1513,"thread_start_us":1361,"threads_started":1}
I20260811 09:01:04.675055 11275 raft_consensus.cc:348] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.675628 11275 raft_consensus.cc:374] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.675738 11275 raft_consensus.cc:729] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb8e6756134d71af8b6f84299c9876, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.676829 11275 consensus_queue.cc:260] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.677807 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c2910df2d64615a0aaefaebd1cb51c" candidate_uuid: "73c07d17aca8431c98e83e39d573d9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb8e6756134d71af8b6f84299c9876" is_pre_election: true
I20260811 09:01:04.678395 11038 raft_consensus.cc:2454] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c07d17aca8431c98e83e39d573d9fd in term 0.
I20260811 09:01:04.679176 10796 leader_election.cc:302] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73c07d17aca8431c98e83e39d573d9fd, d4bb8e6756134d71af8b6f84299c9876; no voters:
I20260811 09:01:04.679621 11278 raft_consensus.cc:2781] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:04.679762 11278 raft_consensus.cc:482] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:04.679908 11278 raft_consensus.cc:3037] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:04.681116 11274 ts_tablet_manager.cc:1428] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent starting tablet: real 0.027s user 0.016s sys 0.005s
I20260811 09:01:04.681583 11251 heartbeater.cc:503] Master 127.9.40.126:33103 was elected leader, sending a full tablet report...
I20260811 09:01:04.683539 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.662590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40397 (local address 127.9.40.67:40119)
0811 09:01:04.662776 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.662782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.675012 (+ 12230us) server_negotiation.cc:407] Connection header received
0811 09:01:04.675310 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.675319 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.675418 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.675491 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:04.676414 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.677511 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.680485 (+ 2974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.682086 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.682218 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.682342 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.683049 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.683211 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:01:04.683861 11278 raft_consensus.cc:504] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.684497 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.662378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40119 (local address 127.9.40.65:40397)
0811 09:01:04.674890 (+ 12512us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:04.674914 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:04.675137 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:04.675951 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:04.675959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:04.675997 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:04.676232 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.676245 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.677769 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.677775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.680228 (+ 2453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:04.680244 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.682602 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.682608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:04.682838 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.682874 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:01:04.683896 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0811 09:01:04.684078 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12356,"thread_start_us":13873,"threads_started":1}
I20260811 09:01:04.684782 11278 leader_election.cc:288] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [CANDIDATE]: Term 1 election: Requested vote from peers d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079), 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119)
I20260811 09:01:04.685534 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c2910df2d64615a0aaefaebd1cb51c" candidate_uuid: "73c07d17aca8431c98e83e39d573d9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb8e6756134d71af8b6f84299c9876"
I20260811 09:01:04.685796 11038 raft_consensus.cc:3037] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:04.686626 11196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c2910df2d64615a0aaefaebd1cb51c" candidate_uuid: "73c07d17aca8431c98e83e39d573d9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" is_pre_election: true
I20260811 09:01:04.686983 11195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89c2910df2d64615a0aaefaebd1cb51c" candidate_uuid: "73c07d17aca8431c98e83e39d573d9fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc"
I20260811 09:01:04.687371 11196 raft_consensus.cc:2454] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73c07d17aca8431c98e83e39d573d9fd in term 0.
I20260811 09:01:04.689723 11038 raft_consensus.cc:2454] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73c07d17aca8431c98e83e39d573d9fd in term 1.
I20260811 09:01:04.690297 10796 leader_election.cc:302] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 73c07d17aca8431c98e83e39d573d9fd, d4bb8e6756134d71af8b6f84299c9876; no voters: 3fafbc9fe1034cb8a9dba365b5eea8cc
I20260811 09:01:04.690623 11278 raft_consensus.cc:2781] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:04.691015 11278 raft_consensus.cc:686] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 1 LEADER]: Becoming Leader. State: Replica: 73c07d17aca8431c98e83e39d573d9fd, State: Running, Role: LEADER
W20260811 09:01:04.691107 10934 tablet.cc:2369] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:04.691687 11278 consensus_queue.cc:237] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.692350 11092 heartbeater.cc:503] Master 127.9.40.126:33103 was elected leader, sending a full tablet report...
I20260811 09:01:04.693342 11275 ts_tablet_manager.cc:1428] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Time spent starting tablet: real 0.032s user 0.014s sys 0.016s
I20260811 09:01:04.697377 10699 catalog_manager.cc:5495] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65). New cstate: current_term: 1 leader_uuid: "73c07d17aca8431c98e83e39d573d9fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } health_report { overall_health: UNKNOWN } } }
W20260811 09:01:04.721412 11093 tablet.cc:2369] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:04.735450 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:04.729426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.9.40.65:44919)
0811 09:01:04.729613 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:04.729630 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:04.729866 (+ 236us) server_negotiation.cc:407] Connection header received
0811 09:01:04.730198 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:04.730201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:04.730300 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:04.730415 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:04.730766 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.731612 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.733311 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:04.733705 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:04.733799 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:04.734329 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:04.734907 (+ 578us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:04.735015 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:04.735173 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 09:01:04.735240 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:01:04.741895 11252 tablet.cc:2369] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:04.744928 11038 raft_consensus.cc:1261] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Refusing update from remote peer 73c07d17aca8431c98e83e39d573d9fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:04.745585 11195 raft_consensus.cc:3037] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:04.745941 11278 consensus_queue.cc:1035] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:04.748603 11195 raft_consensus.cc:1261] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Refusing update from remote peer 73c07d17aca8431c98e83e39d573d9fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:04.749349 11278 consensus_queue.cc:1035] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:04.757905 11290 mvcc.cc:205] Tried to move back new op lower bound from 7317253589982248960 to 7317253589783973888. Current Snapshot: MvccSnapshot[applied={T|T < 7317253589982248960}]
I20260811 09:01:04.780007 10700 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:04.780323 10700 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:04.784982 10700 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Sending DeleteTablet for 3 replicas of tablet 89c2910df2d64615a0aaefaebd1cb51c
I20260811 09:01:04.786264 10855 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:04 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37784
I20260811 09:01:04.786326 11011 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:04 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58020
I20260811 09:01:04.787379 11298 tablet_replica.cc:330] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: stopping tablet replica
I20260811 09:01:04.787608 11170 tablet_service.cc:1514] Processing DeleteTablet for tablet 89c2910df2d64615a0aaefaebd1cb51c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:04 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:60616
I20260811 09:01:04.788043 11298 raft_consensus.cc:2229] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:04.788913 11298 raft_consensus.cc:2258] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:04.791034 11300 tablet_replica.cc:330] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: stopping tablet replica
I20260811 09:01:04.791581 11300 raft_consensus.cc:2229] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:04.792227 11300 raft_consensus.cc:2258] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:04.793107 10700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:04.793833 10700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:04.794674 11300 ts_tablet_manager.cc:1905] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:04.798153 11298 ts_tablet_manager.cc:1905] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:04.802333 11300 ts_tablet_manager.cc:1918] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:04.802507 11300 log.cc:1196] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/89c2910df2d64615a0aaefaebd1cb51c
I20260811 09:01:04.802529 11010 tablet_service.cc:1467] Processing CreateTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 (DEFAULT_TABLE table=test-table [id=757877cbda5a4ac6865ad59a49d7abe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.803222 11300 ts_tablet_manager.cc:1939] T 89c2910df2d64615a0aaefaebd1cb51c P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting consensus metadata
I20260811 09:01:04.803277 11010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c28ab676434e648e3d67bb13a87cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.805630 10855 tablet_service.cc:1467] Processing CreateTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 (DEFAULT_TABLE table=test-table [id=757877cbda5a4ac6865ad59a49d7abe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.806087 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c28ab676434e648e3d67bb13a87cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.809572 10683 catalog_manager.cc:4841] TS 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119): tablet 89c2910df2d64615a0aaefaebd1cb51c (table test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]) successfully deleted
I20260811 09:01:04.810412 11170 tablet_service.cc:1467] Processing CreateTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 (DEFAULT_TABLE table=test-table [id=757877cbda5a4ac6865ad59a49d7abe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:04.811094 11170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c28ab676434e648e3d67bb13a87cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:04.812192 11298 ts_tablet_manager.cc:1918] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:04.812521 11298 log.cc:1196] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/89c2910df2d64615a0aaefaebd1cb51c
I20260811 09:01:04.813350 11298 ts_tablet_manager.cc:1939] T 89c2910df2d64615a0aaefaebd1cb51c P 73c07d17aca8431c98e83e39d573d9fd: Deleting consensus metadata
I20260811 09:01:04.814509 11299 tablet_replica.cc:330] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: stopping tablet replica
I20260811 09:01:04.815008 11299 raft_consensus.cc:2229] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:04.815603 11299 raft_consensus.cc:2258] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:04.815564 10684 catalog_manager.cc:4841] TS 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919): tablet 89c2910df2d64615a0aaefaebd1cb51c (table test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]) successfully deleted
I20260811 09:01:04.818877 11274 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Bootstrap starting.
I20260811 09:01:04.820444 11272 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Bootstrap starting.
I20260811 09:01:04.820670 11274 tablet_bootstrap.cc:654] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.822333 11272 tablet_bootstrap.cc:654] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.823410 11274 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: No bootstrap required, opened a new log
I20260811 09:01:04.823580 11274 ts_tablet_manager.cc:1397] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260811 09:01:04.824246 11299 ts_tablet_manager.cc:1905] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:04.824996 11274 raft_consensus.cc:348] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.825217 11272 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: No bootstrap required, opened a new log
I20260811 09:01:04.825276 11274 raft_consensus.cc:374] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.825369 11272 ts_tablet_manager.cc:1397] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 09:01:04.825438 11274 raft_consensus.cc:729] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fafbc9fe1034cb8a9dba365b5eea8cc, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.825894 11274 consensus_queue.cc:260] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.826505 11274 ts_tablet_manager.cc:1428] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:04.826788 11272 raft_consensus.cc:348] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.827082 11272 raft_consensus.cc:374] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.827203 11272 raft_consensus.cc:729] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c07d17aca8431c98e83e39d573d9fd, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.827553 11272 consensus_queue.cc:260] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.829777 11272 ts_tablet_manager.cc:1428] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 09:01:04.833393 11275 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Bootstrap starting.
I20260811 09:01:04.835170 11275 tablet_bootstrap.cc:654] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:04.837529 11275 tablet_bootstrap.cc:492] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: No bootstrap required, opened a new log
I20260811 09:01:04.837692 11275 ts_tablet_manager.cc:1397] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20260811 09:01:04.838008 11299 ts_tablet_manager.cc:1918] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:04.838191 11299 log.cc:1196] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/89c2910df2d64615a0aaefaebd1cb51c
I20260811 09:01:04.838518 11275 raft_consensus.cc:348] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.838722 11275 raft_consensus.cc:374] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:04.838856 11275 raft_consensus.cc:729] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb8e6756134d71af8b6f84299c9876, State: Initialized, Role: FOLLOWER
I20260811 09:01:04.838997 11299 ts_tablet_manager.cc:1939] T 89c2910df2d64615a0aaefaebd1cb51c P d4bb8e6756134d71af8b6f84299c9876: Deleting consensus metadata
I20260811 09:01:04.839239 11275 consensus_queue.cc:260] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:04.840143 11275 ts_tablet_manager.cc:1428] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 09:01:04.841223 10683 catalog_manager.cc:4841] TS d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079): tablet 89c2910df2d64615a0aaefaebd1cb51c (table test-table [id=52d85fd6168f40c8afdcfc5d2d0855eb]) successfully deleted
I20260811 09:01:05.024202 11285 raft_consensus.cc:482] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:05.024493 11285 raft_consensus.cc:504] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.026144 11285 leader_election.cc:288] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919), 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119)
I20260811 09:01:05.035226 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.027086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56595 (local address 127.9.40.65:44919)
0811 09:01:05.027283 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.027290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.027943 (+ 653us) server_negotiation.cc:407] Connection header received
0811 09:01:05.028136 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.028144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.028220 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.028315 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:05.029158 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.030217 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.032654 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.033916 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.034072 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.034191 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.034741 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.034882 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:01:05.035408 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.026648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52045 (local address 127.9.40.67:40119)
0811 09:01:05.026888 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.026898 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.027593 (+ 695us) server_negotiation.cc:407] Connection header received
0811 09:01:05.027947 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.027954 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.028068 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.028179 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:05.028900 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.029963 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.033167 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.034442 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.034595 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.034692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.035004 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.035147 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:01:05.035537 11304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.026393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40119 (local address 127.9.40.66:52045)
0811 09:01:05.027434 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.027458 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.027754 (+ 296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.028195 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.028200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.028224 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:05.028659 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.028674 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.030107 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.030113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.032873 (+ 2760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.032889 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.034657 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.034663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.034857 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.034878 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.034979 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.035130 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":27,"thread_start_us":754,"threads_started":1}
I20260811 09:01:05.036019 11307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.026446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:44919 (local address 127.9.40.66:56595)
0811 09:01:05.027841 (+ 1395us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.027857 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.027972 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.028396 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.028402 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.028434 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:05.028985 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.028998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.030343 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.030350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.032447 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.032495 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.034273 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.034280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.034527 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.034565 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.035474 (+ 909us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.035617 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1256,"mutex_wait_us":3,"thread_start_us":1020,"threads_started":1}
I20260811 09:01:05.036206 11195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c28ab676434e648e3d67bb13a87cc0" candidate_uuid: "d4bb8e6756134d71af8b6f84299c9876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" is_pre_election: true
I20260811 09:01:05.036484 11195 raft_consensus.cc:2454] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb8e6756134d71af8b6f84299c9876 in term 0.
I20260811 09:01:05.037034 10956 leader_election.cc:302] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fafbc9fe1034cb8a9dba365b5eea8cc, d4bb8e6756134d71af8b6f84299c9876; no voters:
I20260811 09:01:05.037021 10878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c28ab676434e648e3d67bb13a87cc0" candidate_uuid: "d4bb8e6756134d71af8b6f84299c9876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c07d17aca8431c98e83e39d573d9fd" is_pre_election: true
I20260811 09:01:05.037492 11285 raft_consensus.cc:2781] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:05.037606 11285 raft_consensus.cc:482] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:05.037513 10878 raft_consensus.cc:2454] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bb8e6756134d71af8b6f84299c9876 in term 0.
I20260811 09:01:05.037707 11285 raft_consensus.cc:3037] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.041038 11285 raft_consensus.cc:504] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.041772 11285 leader_election.cc:288] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [CANDIDATE]: Term 1 election: Requested vote from peers 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919), 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119)
I20260811 09:01:05.042214 11195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c28ab676434e648e3d67bb13a87cc0" candidate_uuid: "d4bb8e6756134d71af8b6f84299c9876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc"
I20260811 09:01:05.042212 10878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c28ab676434e648e3d67bb13a87cc0" candidate_uuid: "d4bb8e6756134d71af8b6f84299c9876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c07d17aca8431c98e83e39d573d9fd"
I20260811 09:01:05.042448 11195 raft_consensus.cc:3037] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.042444 10878 raft_consensus.cc:3037] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.044507 11195 raft_consensus.cc:2454] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb8e6756134d71af8b6f84299c9876 in term 1.
I20260811 09:01:05.044930 10956 leader_election.cc:302] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fafbc9fe1034cb8a9dba365b5eea8cc, d4bb8e6756134d71af8b6f84299c9876; no voters:
I20260811 09:01:05.045292 11285 raft_consensus.cc:2781] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:05.045957 11285 raft_consensus.cc:686] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 1 LEADER]: Becoming Leader. State: Replica: d4bb8e6756134d71af8b6f84299c9876, State: Running, Role: LEADER
I20260811 09:01:05.046434 11285 consensus_queue.cc:237] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.046782 10878 raft_consensus.cc:2454] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bb8e6756134d71af8b6f84299c9876 in term 1.
I20260811 09:01:05.051244 10699 catalog_manager.cc:5495] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "d4bb8e6756134d71af8b6f84299c9876" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438862, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:05.102403 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.087101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36732 (local address 127.9.40.126:33103)
0811 09:01:05.087941 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.087949 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.088241 (+ 292us) server_negotiation.cc:407] Connection header received
0811 09:01:05.088454 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.088458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.088513 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.088626 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.089961 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.090743 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.092030 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.092444 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.092582 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.096392 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.096397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.096400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.096428 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.099211 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.100177 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.100181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.100183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.100306 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.100812 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.100815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.100817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.101057 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.101178 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.101766 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.102129 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":164,"server-negotiator.queue_time_us":649,"thread_start_us":382,"threads_started":1}
I20260811 09:01:05.102531 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.087215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33103 (local address 127.0.0.1:36732)
0811 09:01:05.088139 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.088154 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.088321 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.088654 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.088658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.089460 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:05.089737 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.089747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.090881 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.090886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.091891 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.091903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.092634 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.092641 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.092870 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.093534 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:05.093551 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:05.096174 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:05.099353 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:05.099366 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:05.099376 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:05.100036 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:05.100441 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:05.100448 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:05.100452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:05.100662 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:05.101209 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:05.101219 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:05.101476 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.101666 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.101987 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":648,"threads_started":1}
I20260811 09:01:05.108314 10699 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:36732:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:01:05.110051 10699 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:05.118465 11010 tablet_service.cc:1467] Processing CreateTablet for tablet 3941206fde9947bfb5ba4d7535a3fab1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91c0a998b055425192a97784f8351324]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:05.118489 11170 tablet_service.cc:1467] Processing CreateTablet for tablet 3941206fde9947bfb5ba4d7535a3fab1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91c0a998b055425192a97784f8351324]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:05.119063 11170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3941206fde9947bfb5ba4d7535a3fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:05.119078 11010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3941206fde9947bfb5ba4d7535a3fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:05.117729 10855 tablet_service.cc:1467] Processing CreateTablet for tablet 3941206fde9947bfb5ba4d7535a3fab1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=91c0a998b055425192a97784f8351324]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:05.121279 10855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3941206fde9947bfb5ba4d7535a3fab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:05.127040 11274 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Bootstrap starting.
I20260811 09:01:05.127040 11275 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: Bootstrap starting.
I20260811 09:01:05.128959 11275 tablet_bootstrap.cc:654] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:05.129186 11274 tablet_bootstrap.cc:654] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:05.133267 11272 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: Bootstrap starting.
I20260811 09:01:05.135329 11272 tablet_bootstrap.cc:654] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:05.139642 11274 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: No bootstrap required, opened a new log
I20260811 09:01:05.139938 11274 ts_tablet_manager.cc:1397] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.001s
I20260811 09:01:05.140488 11272 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: No bootstrap required, opened a new log
I20260811 09:01:05.140682 11272 ts_tablet_manager.cc:1397] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260811 09:01:05.140899 11274 raft_consensus.cc:348] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.141149 11274 raft_consensus.cc:374] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:05.141278 11274 raft_consensus.cc:729] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fafbc9fe1034cb8a9dba365b5eea8cc, State: Initialized, Role: FOLLOWER
I20260811 09:01:05.141611 11272 raft_consensus.cc:348] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.141628 11274 consensus_queue.cc:260] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.141867 11272 raft_consensus.cc:374] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:05.141965 11272 raft_consensus.cc:729] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c07d17aca8431c98e83e39d573d9fd, State: Initialized, Role: FOLLOWER
I20260811 09:01:05.142270 11284 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.142449 11272 consensus_queue.cc:260] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.142699 11284 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.142987 11277 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.143333 11277 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.143893 11272 ts_tablet_manager.cc:1428] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:05.145166 11274 ts_tablet_manager.cc:1428] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 09:01:05.146720 11275 tablet_bootstrap.cc:492] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: No bootstrap required, opened a new log
I20260811 09:01:05.146906 11275 ts_tablet_manager.cc:1397] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.007s
I20260811 09:01:05.147806 11275 raft_consensus.cc:348] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.148099 11275 raft_consensus.cc:374] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:05.148204 11275 raft_consensus.cc:729] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bb8e6756134d71af8b6f84299c9876, State: Initialized, Role: FOLLOWER
I20260811 09:01:05.148487 11275 consensus_queue.cc:260] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.149269 11275 ts_tablet_manager.cc:1428] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:01:05.149608 11285 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.149950 11285 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.235162 11284 raft_consensus.cc:482] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:05.235486 11284 raft_consensus.cc:504] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.237299 11284 leader_election.cc:288] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919), d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079)
I20260811 09:01:05.255072 11321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.237586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45079 (local address 127.9.40.67:54477)
0811 09:01:05.248786 (+ 11200us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.248809 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.248956 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.249436 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.249442 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.249477 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:05.249917 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.249928 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.251135 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.251142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.252861 (+ 1719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.252875 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.254150 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.254156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.254358 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.254405 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.254528 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.254684 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10943,"thread_start_us":10990,"threads_started":1}
I20260811 09:01:05.256218 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.237790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54477 (local address 127.9.40.66:45079)
0811 09:01:05.247925 (+ 10135us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.247935 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.248934 (+ 999us) server_negotiation.cc:407] Connection header received
0811 09:01:05.249105 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.249112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.249223 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.249324 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:05.250078 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.250986 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.253039 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.253958 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.254099 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.254230 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.255708 (+ 1478us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.255874 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9964,"thread_start_us":9836,"threads_started":1}
I20260811 09:01:05.257124 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.237930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:47209 (local address 127.9.40.65:44919)
0811 09:01:05.238141 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.238151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.248469 (+ 10318us) server_negotiation.cc:407] Connection header received
0811 09:01:05.248923 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.248930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.249085 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.249198 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:05.249864 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.251082 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.253957 (+ 2875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.255320 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.255474 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.255600 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.256600 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.256788 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:01:05.257182 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3941206fde9947bfb5ba4d7535a3fab1" candidate_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb8e6756134d71af8b6f84299c9876" is_pre_election: true
I20260811 09:01:05.257526 11038 raft_consensus.cc:2454] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fafbc9fe1034cb8a9dba365b5eea8cc in term 0.
I20260811 09:01:05.257917 11320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.237506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:44919 (local address 127.9.40.67:47209)
0811 09:01:05.248271 (+ 10765us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.248308 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.248624 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.249231 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.249236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.249267 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:05.249590 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.249605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.251271 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.251276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.253755 (+ 2479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.253773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.256151 (+ 2378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.256159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.256369 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.256404 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.257390 (+ 986us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.257537 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10600,"thread_start_us":10413,"threads_started":1}
I20260811 09:01:05.258370 11116 leader_election.cc:302] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fafbc9fe1034cb8a9dba365b5eea8cc, d4bb8e6756134d71af8b6f84299c9876; no voters:
I20260811 09:01:05.258651 10878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3941206fde9947bfb5ba4d7535a3fab1" candidate_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c07d17aca8431c98e83e39d573d9fd" is_pre_election: true
I20260811 09:01:05.258850 11284 raft_consensus.cc:2781] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:05.259001 11284 raft_consensus.cc:482] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:05.258978 10878 raft_consensus.cc:2454] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3fafbc9fe1034cb8a9dba365b5eea8cc in term 0.
I20260811 09:01:05.259109 11284 raft_consensus.cc:3037] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.261785 11284 raft_consensus.cc:504] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.262458 11284 leader_election.cc:288] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [CANDIDATE]: Term 1 election: Requested vote from peers 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919), d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079)
I20260811 09:01:05.262996 10878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3941206fde9947bfb5ba4d7535a3fab1" candidate_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73c07d17aca8431c98e83e39d573d9fd"
I20260811 09:01:05.263037 11038 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3941206fde9947bfb5ba4d7535a3fab1" candidate_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4bb8e6756134d71af8b6f84299c9876"
I20260811 09:01:05.263237 10878 raft_consensus.cc:3037] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.263260 11038 raft_consensus.cc:3037] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:05.265556 10878 raft_consensus.cc:2454] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fafbc9fe1034cb8a9dba365b5eea8cc in term 1.
I20260811 09:01:05.266057 11118 leader_election.cc:302] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fafbc9fe1034cb8a9dba365b5eea8cc, 73c07d17aca8431c98e83e39d573d9fd; no voters:
I20260811 09:01:05.266395 11284 raft_consensus.cc:2781] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:05.266372 11038 raft_consensus.cc:2454] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3fafbc9fe1034cb8a9dba365b5eea8cc in term 1.
I20260811 09:01:05.266973 11284 raft_consensus.cc:686] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 LEADER]: Becoming Leader. State: Replica: 3fafbc9fe1034cb8a9dba365b5eea8cc, State: Running, Role: LEADER
I20260811 09:01:05.267460 11284 consensus_queue.cc:237] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } }
I20260811 09:01:05.269707 11324 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: TxnStatusTablet state changed. Reason: New leader 3fafbc9fe1034cb8a9dba365b5eea8cc. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.270076 11324 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:05.271709 11326 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:01:05.272006 11326 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:01:05.271893 10698 catalog_manager.cc:5495] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67). New cstate: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } health_report { overall_health: HEALTHY } } }
I20260811 09:01:05.303714 11316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.296783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40119 (local address 127.0.0.1:60628)
0811 09:01:05.297019 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.297040 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.297213 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.297693 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.297699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.297713 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:05.297933 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.297945 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.299182 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.299188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.300899 (+ 1711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.300916 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.301778 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.301786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.302039 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.302113 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.303028 (+ 915us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.303059 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.303157 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.303334 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 09:01:05.303767 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.297056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60628 (local address 127.9.40.67:40119)
0811 09:01:05.297264 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.297270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.297289 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:05.297435 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.297439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.297520 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.297643 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:05.298201 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.299032 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.301053 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.301604 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.301717 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.302309 (+ 592us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.302885 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.303017 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.303257 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.303381 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:01:05.308815 11038 raft_consensus.cc:1261] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Refusing update from remote peer 3fafbc9fe1034cb8a9dba365b5eea8cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:05.309177 10878 raft_consensus.cc:1261] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Refusing update from remote peer 3fafbc9fe1034cb8a9dba365b5eea8cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:05.309615 11284 consensus_queue.cc:1035] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:05.310016 11324 consensus_queue.cc:1035] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:05.313997 11285 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: TxnStatusTablet state changed. Reason: New leader 3fafbc9fe1034cb8a9dba365b5eea8cc. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.314489 11285 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.318954 11284 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.319314 11284 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:05.326467 11285 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.326915 11285 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P d4bb8e6756134d71af8b6f84299c9876: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.326730 11277 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: TxnStatusTablet state changed. Reason: New leader 3fafbc9fe1034cb8a9dba365b5eea8cc. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.327114 11277 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.329079 11324 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.329367 11324 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 3fafbc9fe1034cb8a9dba365b5eea8cc: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:05.337606 11277 tablet_replica.cc:439] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 } } peers { permanent_uuid: "d4bb8e6756134d71af8b6f84299c9876" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45079 } } peers { permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 } } }
I20260811 09:01:05.337911 11277 tablet_replica.cc:442] T 3941206fde9947bfb5ba4d7535a3fab1 P 73c07d17aca8431c98e83e39d573d9fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:05.347134 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.340105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58030 (local address 127.9.40.66:45079)
0811 09:01:05.340303 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.340310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.340791 (+ 481us) server_negotiation.cc:407] Connection header received
0811 09:01:05.341020 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.341026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.341176 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.341363 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:05.341993 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.343114 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.344439 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.345193 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.345337 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.345747 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.346489 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.346643 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.346731 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.346822 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:01:05.364804 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.358078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:54087 (local address 127.9.40.67:40119)
0811 09:01:05.358301 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.358308 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.358840 (+ 532us) server_negotiation.cc:407] Connection header received
0811 09:01:05.359049 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.359053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.359114 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.359194 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:05.359764 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.360525 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.362709 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.363321 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.363437 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.364057 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.364179 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.364248 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.364420 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.364485 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:01:05.365134 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.357862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40119 (local address 127.9.40.66:54087)
0811 09:01:05.358726 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.358748 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.358917 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.359238 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.359245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.359258 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:05.359617 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.359629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.360670 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.360677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.362531 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.362548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.363535 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.363543 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.363762 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.363839 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.364283 (+ 444us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.364296 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.364395 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.364562 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"thread_start_us":551,"threads_started":1}
I20260811 09:01:05.385262 10877 raft_consensus.cc:1261] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Refusing update from remote peer d4bb8e6756134d71af8b6f84299c9876: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:05.385355 11195 raft_consensus.cc:1261] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Refusing update from remote peer d4bb8e6756134d71af8b6f84299c9876: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:05.386003 11308 consensus_queue.cc:1035] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73c07d17aca8431c98e83e39d573d9fd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 44919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:05.386471 11285 consensus_queue.cc:1035] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fafbc9fe1034cb8a9dba365b5eea8cc" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:05.431084 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.423607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:42813 (local address 127.9.40.66:45079)
0811 09:01:05.423776 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.423780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.425257 (+ 1477us) server_negotiation.cc:407] Connection header received
0811 09:01:05.425557 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.425563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.425646 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.425741 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:05.426349 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.427205 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.429168 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.429617 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.429756 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.430331 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.430446 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.430567 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.430691 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.430771 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:01:05.431222 11347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.424291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45079 (local address 127.9.40.67:42813)
0811 09:01:05.425135 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:05.425158 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:05.425347 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:05.425756 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:05.425761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:05.425772 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:05.426187 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.426197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.427356 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.427369 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.429044 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:05.429059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.429827 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.429836 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:05.430083 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.430138 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:05.430569 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:05.430587 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 09:01:05.430661 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 09:01:05.430802 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":677,"threads_started":1}
I20260811 09:01:05.476744 10698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:05.477051 10698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:05.481133 10698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bae197e7ec5a49e2b8b17d6ab5028c93: Sending DeleteTablet for 3 replicas of tablet a3c28ab676434e648e3d67bb13a87cc0
I20260811 09:01:05.482394 11010 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58020
I20260811 09:01:05.489537 10855 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37784
I20260811 09:01:05.491451 11170 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c28ab676434e648e3d67bb13a87cc0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:60616
I20260811 09:01:05.493029 11352 tablet_replica.cc:330] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: stopping tablet replica
I20260811 09:01:05.493265 11352 raft_consensus.cc:2229] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:05.493899 11352 raft_consensus.cc:2258] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:05.495023 11352 ts_tablet_manager.cc:1905] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:05.503007 11352 ts_tablet_manager.cc:1918] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:05.503151 11352 log.cc:1196] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/a3c28ab676434e648e3d67bb13a87cc0
I20260811 09:01:05.503682 11352 ts_tablet_manager.cc:1939] T a3c28ab676434e648e3d67bb13a87cc0 P d4bb8e6756134d71af8b6f84299c9876: Deleting consensus metadata
I20260811 09:01:05.504861 10683 catalog_manager.cc:4841] TS d4bb8e6756134d71af8b6f84299c9876 (127.9.40.66:45079): tablet a3c28ab676434e648e3d67bb13a87cc0 (table test-table [id=757877cbda5a4ac6865ad59a49d7abe6]) successfully deleted
I20260811 09:01:05.509552 11354 tablet_replica.cc:330] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: stopping tablet replica
I20260811 09:01:05.509775 11354 raft_consensus.cc:2229] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:05.510144 11354 raft_consensus.cc:2258] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:05.510814 11353 tablet_replica.cc:330] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: stopping tablet replica
I20260811 09:01:05.511063 11353 raft_consensus.cc:2229] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:05.511333 11353 raft_consensus.cc:2258] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:05.514487 11354 ts_tablet_manager.cc:1905] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:05.515081 11353 ts_tablet_manager.cc:1905] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:05.536109 11354 ts_tablet_manager.cc:1918] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:05.536296 11354 log.cc:1196] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/a3c28ab676434e648e3d67bb13a87cc0
I20260811 09:01:05.536851 11354 ts_tablet_manager.cc:1939] T a3c28ab676434e648e3d67bb13a87cc0 P 3fafbc9fe1034cb8a9dba365b5eea8cc: Deleting consensus metadata
I20260811 09:01:05.538169 10683 catalog_manager.cc:4841] TS 3fafbc9fe1034cb8a9dba365b5eea8cc (127.9.40.67:40119): tablet a3c28ab676434e648e3d67bb13a87cc0 (table test-table [id=757877cbda5a4ac6865ad59a49d7abe6]) successfully deleted
I20260811 09:01:05.542762 11353 ts_tablet_manager.cc:1918] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:05.542938 11353 log.cc:1196] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/a3c28ab676434e648e3d67bb13a87cc0
I20260811 09:01:05.543412 11353 ts_tablet_manager.cc:1939] T a3c28ab676434e648e3d67bb13a87cc0 P 73c07d17aca8431c98e83e39d573d9fd: Deleting consensus metadata
I20260811 09:01:05.544523 10684 catalog_manager.cc:4841] TS 73c07d17aca8431c98e83e39d573d9fd (127.9.40.65:44919): tablet a3c28ab676434e648e3d67bb13a87cc0 (table test-table [id=757877cbda5a4ac6865ad59a49d7abe6]) successfully deleted
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:05.559669 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.511904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37810 (local address 127.9.40.65:44919)
0811 09:01:05.512129 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.512137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.529963 (+ 17826us) server_negotiation.cc:407] Connection header received
0811 09:01:05.530523 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.530529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.530611 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.530679 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.539008 (+ 8329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.540099 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.544864 (+ 4765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.545618 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.545721 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.551412 (+ 5691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.551430 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.551448 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.551489 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.555642 (+ 4153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.556551 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.556561 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.556564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.556757 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.557442 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.557448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.557452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.557920 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.558341 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.559007 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.559232 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 09:01:05.561451 10927 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37810
I20260811 09:01:05.586802 11315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.566341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.9.40.126:33103)
0811 09:01:05.566544 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.566550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.569281 (+ 2731us) server_negotiation.cc:407] Connection header received
0811 09:01:05.569534 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.569541 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.569635 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.569749 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.572252 (+ 2503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.573479 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.575423 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.576238 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.576381 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.579527 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.579534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.579540 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.579592 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.583681 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.584488 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.584494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.584498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.584655 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.585094 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.585101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.585105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.585489 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.585742 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.586167 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.586433 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 09:01:05.587559 10698 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36734
I20260811 09:01:05.590636 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10780
I20260811 09:01:05.613965 11265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.592687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.9.40.65:44919)
0811 09:01:05.592938 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.592945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.597272 (+ 4327us) server_negotiation.cc:407] Connection header received
0811 09:01:05.597411 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.597418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.597524 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.597659 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.599300 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.600501 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.602490 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.603200 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.603347 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.606480 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.606489 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.606494 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.606535 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.610394 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.611304 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.611310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.611314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.611505 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.611979 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.611985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.611989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.612404 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.612736 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.613168 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.613458 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 09:01:05.614677 10927 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37818
W20260811 09:01:05.615412 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10780: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:05.615648 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10780
I20260811 09:01:05.657025 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10938
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:05.679065 11323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.657960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58032 (local address 127.9.40.66:45079)
0811 09:01:05.658174 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.658181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.658206 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:01:05.658417 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.658429 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.658531 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.658654 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.660473 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.661723 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.663847 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.664540 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.664711 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.670446 (+ 5735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.670463 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.670485 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.670535 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.675050 (+ 4515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.676020 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.676029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.676033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.676235 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.676753 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.676758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.676762 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.677371 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.677671 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.678429 (+ 758us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.678676 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 09:01:05.679826 11086 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58032
W20260811 09:01:05.680596 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10938: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:05.680845 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10938
I20260811 09:01:05.702268 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11096
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[10648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:05.720227 11271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:05.703260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60630 (local address 127.9.40.67:40119)
0811 09:01:05.703499 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:05.703506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:05.703531 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:01:05.703603 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:05.703608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:05.703695 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:05.703809 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:05.705432 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.706717 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.708111 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:05.708910 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:05.709081 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:05.713156 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:05.713170 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:05.713191 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:05.713248 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:05.717447 (+ 4199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.718114 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.718130 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.718133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.718275 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:05.718681 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:05.718687 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:05.718690 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:05.719034 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:05.719403 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:05.719656 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 09:01:05.719881 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 09:01:05.721042 11246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60630
W20260811 09:01:05.721884 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11096: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:05.722043 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 11096
I20260811 09:01:05.747064 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 10662
I20260811 09:01:05.748487 10745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:05.866094 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 10662
2026-08-11T09:01:05Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4887 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[11367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[11367](info): set up 2 sockets
Aug 11 09:01:05 dist-test-slave-cbsf krb5kdc[11367](info): commencing operation
krb5kdc: starting...
W20260811 09:01:07.034688 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:07 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:07.233979 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36749
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:07.553781 11380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:07.554104 11380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:07.554272 11380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:07.561265 11380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:07.561404 11380 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:01:07.561486 11380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:07.561578 11380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:07.561667 11380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:07.561754 11380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:07.571182 11380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36749
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:07.572563 11380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:07.573902 11380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:07.585613 11387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:07.585537 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:07.585613 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:07.586494 11380 server_base.cc:1034] running on GCE node
I20260811 09:01:07.587184 11380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:07.588626 11380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:07.589876 11380 hybrid_clock.cc:648] HybridClock initialized: now 1786438867589810 us; error 48 us; skew 500 ppm
I20260811 09:01:07.590204 11380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:07 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:07.593410 11380 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:07.594831 11380 webserver.cc:460] Webserver started at http://127.9.40.126:41373/ using document root <none> and password file <none>
I20260811 09:01:07.595513 11380 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:07.595638 11380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:07.595981 11380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:07.599028 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "5f44589154fa461eb057ef57c77ef37a"
format_stamp: "Formatted at 2026-08-11 09:01:07 on dist-test-slave-cbsf"
server_key: "80090a8d5c21a99ca5dc2283b7b5d5c5"
server_key_iv: "315e037c15bd4bd6d79a1c90a37e4c93"
server_key_version: "encryptionkey@0"
I20260811 09:01:07.599797 11380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "5f44589154fa461eb057ef57c77ef37a"
format_stamp: "Formatted at 2026-08-11 09:01:07 on dist-test-slave-cbsf"
server_key: "80090a8d5c21a99ca5dc2283b7b5d5c5"
server_key_iv: "315e037c15bd4bd6d79a1c90a37e4c93"
server_key_version: "encryptionkey@0"
I20260811 09:01:07.604864 11380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:01:07.608215 11399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:07.609601 11380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:07.609797 11380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "5f44589154fa461eb057ef57c77ef37a"
format_stamp: "Formatted at 2026-08-11 09:01:07 on dist-test-slave-cbsf"
server_key: "80090a8d5c21a99ca5dc2283b7b5d5c5"
server_key_iv: "315e037c15bd4bd6d79a1c90a37e4c93"
server_key_version: "encryptionkey@0"
I20260811 09:01:07.610005 11380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:07.639483 11380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:07.644042 11380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:07.644317 11380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:07.666145 11380 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36749
I20260811 09:01:07.666152 11469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36749 every 8 connection(s)
I20260811 09:01:07.667477 11380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:07.671056 11470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:07.672641 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 11380
I20260811 09:01:07.672914 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:07.673631 9377 external_mini_cluster.cc:1428] Setting key aa2320a7760b83b68ff608a99d9fffef
I20260811 09:01:07.684203 11470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Bootstrap starting.
I20260811 09:01:07.687829 11470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:07.689013 11470 log.cc:824] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:01:07 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:07.692595 11470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: No bootstrap required, opened a new log
I20260811 09:01:07.698328 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:07.675663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.9.40.126:36749)
0811 09:01:07.676529 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:07.676549 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:07.676597 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:01:07.679096 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:07.679123 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:07.679593 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:07.680009 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:07.681345 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:07.683396 (+ 2051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:07.684767 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:07.685834 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:07.686066 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:07.690527 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:07.690542 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:07.690565 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:07.690624 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:07.694099 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:07.694838 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:07.694845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:07.694852 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:07.695029 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:07.695431 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:07.695436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:07.695449 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:07.696005 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:07.696300 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:07.696817 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 09:01:07.697263 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":348,"threads_started":1}
I20260811 09:01:07.700423 11470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } }
I20260811 09:01:07.700839 11470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:07.700986 11470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f44589154fa461eb057ef57c77ef37a, State: Initialized, Role: FOLLOWER
I20260811 09:01:07.701581 11470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } }
I20260811 09:01:07.701825 11470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:07.701915 11470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:07.702047 11470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:07.704545 11470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } }
I20260811 09:01:07.705042 11470 leader_election.cc:302] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f44589154fa461eb057ef57c77ef37a; no voters:
I20260811 09:01:07.705526 11470 leader_election.cc:288] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:07.705673 11475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:07.706403 11475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [term 1 LEADER]: Becoming Leader. State: Replica: 5f44589154fa461eb057ef57c77ef37a, State: Running, Role: LEADER
I20260811 09:01:07.706979 11475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } }
I20260811 09:01:07.707682 11470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:07.709582 11476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f44589154fa461eb057ef57c77ef37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } } }
I20260811 09:01:07.709909 11476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:07.710793 11482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:07.710619 11477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f44589154fa461eb057ef57c77ef37a. Latest consensus state: current_term: 1 leader_uuid: "5f44589154fa461eb057ef57c77ef37a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f44589154fa461eb057ef57c77ef37a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36749 } } }
I20260811 09:01:07.710979 11477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:07.722824 11482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:07.731967 11482 catalog_manager.cc:1347] Generated new cluster ID: dc372d2095544717807b153ef3a7f974
I20260811 09:01:07.732103 11482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:07.749434 11482 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:07.751216 11482 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:07.764621 11482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Generated new TSK 0
I20260811 09:01:07.765635 11482 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:07.814095 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36749
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:08.137614 11500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:08.137874 11500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:08.138039 11500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:08.145256 11500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:08.145357 11500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:08.145552 11500 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:01:08.145618 11500 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:08.145725 11500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:08.155203 11500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:36749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:08.156569 11500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:08.157963 11500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:08.170943 11505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:08.170990 11508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:08.171034 11506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:08.171893 11500 server_base.cc:1034] running on GCE node
I20260811 09:01:08.172468 11500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:08.173547 11500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:08.174773 11500 hybrid_clock.cc:648] HybridClock initialized: now 1786438868174711 us; error 55 us; skew 500 ppm
I20260811 09:01:08.175084 11500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:08 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438868, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:08.178124 11500 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:08.179644 11500 webserver.cc:460] Webserver started at http://127.9.40.65:41103/ using document root <none> and password file <none>
I20260811 09:01:08.180322 11500 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:08.180477 11500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:08.180867 11500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:08.183779 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "8df479a596ef474e811edf9a387e8ebf"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "322ba0500714f7d8b910996459c1d348"
server_key_iv: "3d2baa68978ffe427e20869a1c42ccfd"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.184528 11500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "8df479a596ef474e811edf9a387e8ebf"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "322ba0500714f7d8b910996459c1d348"
server_key_iv: "3d2baa68978ffe427e20869a1c42ccfd"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.189571 11500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:01:08.192895 11517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:08.194180 11500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:01:08.194370 11500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "8df479a596ef474e811edf9a387e8ebf"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "322ba0500714f7d8b910996459c1d348"
server_key_iv: "3d2baa68978ffe427e20869a1c42ccfd"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.194624 11500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:08.220039 11500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:08.224350 11500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:08.224697 11500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:08.227308 11500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:08.227463 11500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:08.227594 11500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:08.227689 11500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:08 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438868, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:08.253750 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.228275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51415 (local address 127.9.40.126:36749)
0811 09:01:08.228775 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:08.228782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:08.230023 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 09:01:08.233223 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:08.233230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:08.233326 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:08.233468 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:08.235674 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.237033 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.239350 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.240130 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.240300 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.244762 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:08.244772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:08.244779 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:08.244833 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:08.249027 (+ 4194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.250281 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.250288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.250295 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.250460 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.251069 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.251077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.251082 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.251541 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:08.251733 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:08.252877 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0811 09:01:08.253204 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260811 09:01:08.254513 11527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.228920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.65:51415)
0811 09:01:08.229828 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:08.229880 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:08.232812 (+ 2932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:08.233747 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:08.233761 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:08.234575 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:08.235455 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.235478 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.237237 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.237245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.239160 (+ 1915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.239201 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.240432 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.240442 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.240682 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.241486 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:08.241537 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:08.244493 (+ 2956us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:08.249226 (+ 4733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.249240 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.249260 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.250068 (+ 808us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.250609 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.250617 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.250623 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.250946 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.251703 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:08.251715 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:08.252291 (+ 576us) client_negotiation.cc:769] Sending connection context
0811 09:01:08.252666 (+ 375us) client_negotiation.cc:240] Negotiation successful
0811 09:01:08.253175 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":367,"threads_started":1}
I20260811 09:01:08.276671 11500 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:34569
I20260811 09:01:08.276779 11647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:34569 every 8 connection(s)
I20260811 09:01:08.278463 11500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:08.280194 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.262356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35641 (local address 127.9.40.126:36749)
0811 09:01:08.262760 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:08.262767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:08.263204 (+ 437us) server_negotiation.cc:407] Connection header received
0811 09:01:08.263411 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:08.263419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:08.263517 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:08.263626 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:08.265727 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.266980 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.269314 (+ 2334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.270049 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.270199 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.273307 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:08.273317 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:08.273321 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:08.273365 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:08.277071 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.277859 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.277862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.277864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.278009 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.278393 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.278396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.278399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.278762 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:08.278874 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:08.279545 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 09:01:08.279764 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
I20260811 09:01:08.280901 11649 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36749
I20260811 09:01:08.281165 11649 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:08.281527 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.262300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.65:35641)
0811 09:01:08.263086 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:08.263106 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:08.263282 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:08.263928 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:08.263938 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:08.264924 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:08.265529 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.265550 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.267162 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.267170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.269132 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.269153 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.270307 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.270317 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.270580 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.271401 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:08.271431 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:08.273081 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:08.277201 (+ 4120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.277206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.277210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.277741 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.278123 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.278128 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.278131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.278282 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.278880 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:08.278889 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:08.279178 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 09:01:08.280395 (+ 1217us) client_negotiation.cc:240] Negotiation successful
0811 09:01:08.280993 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":491,"threads_started":1}
I20260811 09:01:08.281971 11649 heartbeater.cc:511] Master 127.9.40.126:36749 requested a full tablet report, sending...
I20260811 09:01:08.283842 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 11500
I20260811 09:01:08.284080 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:08.284128 11416 ts_manager.cc:194] Registered new tserver with Master: 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569)
I20260811 09:01:08.285059 9377 external_mini_cluster.cc:1428] Setting key 18018a7a2d3eddf2933ab34e73ebf962
I20260811 09:01:08.286322 11416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:51415
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:08.334347 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36749
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:08.666626 11653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:08.666994 11653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:08.667150 11653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:08.678373 11653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:08.678499 11653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:08.678633 11653 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:01:08.678699 11653 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:08.678795 11653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:08.691969 11653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:36749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:08.693341 11653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:08.694548 11653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:08.707873 11659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:08.707962 11660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:08.707842 11662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:08.714386 11653 server_base.cc:1034] running on GCE node
I20260811 09:01:08.715044 11653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:08.715989 11653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:08.717221 11653 hybrid_clock.cc:648] HybridClock initialized: now 1786438868717180 us; error 48 us; skew 500 ppm
I20260811 09:01:08.717557 11653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:08 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438868, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:08.725353 11653 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:08.727072 11653 webserver.cc:460] Webserver started at http://127.9.40.66:46049/ using document root <none> and password file <none>
I20260811 09:01:08.727854 11653 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:08.728014 11653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:08.728379 11653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:08.731467 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "62fc6e4a85f7474bb74fb0287c30a897"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "925b74f70a15217d2596e0222514b32f"
server_key_iv: "1cc91d574c26d67cafc026622a2d20ef"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.732261 11653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "62fc6e4a85f7474bb74fb0287c30a897"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "925b74f70a15217d2596e0222514b32f"
server_key_iv: "1cc91d574c26d67cafc026622a2d20ef"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.737872 11653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:01:08.741498 11669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:08.742839 11653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:08.743057 11653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "62fc6e4a85f7474bb74fb0287c30a897"
format_stamp: "Formatted at 2026-08-11 09:01:08 on dist-test-slave-cbsf"
server_key: "925b74f70a15217d2596e0222514b32f"
server_key_iv: "1cc91d574c26d67cafc026622a2d20ef"
server_key_version: "encryptionkey@0"
I20260811 09:01:08.743290 11653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:08.759241 11653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:08.763077 11653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:08.763378 11653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:08.766206 11653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:08.766332 11653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:08.766501 11653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:08.766613 11653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:08 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438868, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:08.790345 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.766758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39115 (local address 127.9.40.126:36749)
0811 09:01:08.767015 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:08.767025 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:08.768489 (+ 1464us) server_negotiation.cc:407] Connection header received
0811 09:01:08.771689 (+ 3200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:08.771696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:08.771801 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:08.771914 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:08.774365 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.775832 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.778491 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.778919 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.779092 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.783917 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:08.783940 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:08.783943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:08.783978 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:08.787161 (+ 3183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.787882 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.787886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.787888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.787996 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.788405 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.788409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.788412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.788687 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:08.788875 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:08.789772 (+ 897us) server_negotiation.cc:299] Negotiation successful
0811 09:01:08.789986 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:01:08.791385 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.767390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.66:39115)
0811 09:01:08.768291 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:08.768350 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:08.771242 (+ 2892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:08.772339 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:08.772354 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:08.773278 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:08.774104 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.774126 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.776003 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.776007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.778314 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.778326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.779189 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.779197 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.779593 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.780382 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:08.780426 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:08.783681 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:08.787297 (+ 3616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.787306 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.787321 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.787751 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.788108 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.788114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.788128 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.788291 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.788851 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:08.788859 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:08.789338 (+ 479us) client_negotiation.cc:769] Sending connection context
0811 09:01:08.789681 (+ 343us) client_negotiation.cc:240] Negotiation successful
0811 09:01:08.790189 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"thread_start_us":383,"threads_started":1}
I20260811 09:01:08.815503 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.798467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.66:56133)
0811 09:01:08.799278 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:08.799302 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:08.799482 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:08.799811 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:08.799818 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:08.800558 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:08.801113 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.801133 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.802390 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.802400 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.804563 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:08.804582 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.805719 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.805730 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:08.806009 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.806734 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:08.806749 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:08.807425 (+ 676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:08.811822 (+ 4397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.811828 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.811831 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.812466 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.813038 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:08.813066 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:08.813072 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:08.813219 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:08.814080 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:08.814085 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:08.814303 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 09:01:08.814615 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 09:01:08.815010 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":399,"threads_started":1}
I20260811 09:01:08.816112 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:08.798505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56133 (local address 127.9.40.126:36749)
0811 09:01:08.798721 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:08.798726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:08.799402 (+ 676us) server_negotiation.cc:407] Connection header received
0811 09:01:08.799603 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:08.799607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:08.799665 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:08.799743 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:08.801297 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.802225 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.804775 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:08.805332 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:08.805458 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:08.807642 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:08.807651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:08.807660 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:08.807711 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:08.811687 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.812668 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.812675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.812680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.812889 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:08.813385 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:08.813389 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:08.813391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:08.813953 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:08.814105 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:08.815401 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0811 09:01:08.815697 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:01:08.823870 11653 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:43123
I20260811 09:01:08.823915 11802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:43123 every 8 connection(s)
I20260811 09:01:08.825129 11653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:08.826627 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 11653
I20260811 09:01:08.826817 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:08.827358 9377 external_mini_cluster.cc:1428] Setting key b8715edd203f0b570fbcca080f3e9905
I20260811 09:01:08.827196 11804 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36749
I20260811 09:01:08.828342 11804 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:08.829172 11804 heartbeater.cc:511] Master 127.9.40.126:36749 requested a full tablet report, sending...
I20260811 09:01:08.830510 11416 ts_manager.cc:194] Registered new tserver with Master: 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123)
I20260811 09:01:08.831348 11416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:39115
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:08.873646 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36749
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:09.199281 11809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:09.199550 11809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:09.199657 11809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:09.206511 11809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:09.206619 11809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:09.206702 11809 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:01:09.206745 11809 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:09.206811 11809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:09.216459 11809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:36749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:09.217715 11809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:09.219013 11809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:09.232015 11815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:09.232136 11814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:09.232049 11817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:09.233017 11809 server_base.cc:1034] running on GCE node
I20260811 09:01:09.233597 11809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:09.234498 11809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:09.235733 11809 hybrid_clock.cc:648] HybridClock initialized: now 1786438869235700 us; error 52 us; skew 500 ppm
I20260811 09:01:09.236048 11809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:09 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438869, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:09.239226 11809 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:09.240649 11809 webserver.cc:460] Webserver started at http://127.9.40.67:41977/ using document root <none> and password file <none>
I20260811 09:01:09.241371 11809 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:09.241472 11809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:09.241715 11809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:09.244421 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
format_stamp: "Formatted at 2026-08-11 09:01:09 on dist-test-slave-cbsf"
server_key: "c7f10a9e36ae3e0cf96a920ed3d28ce8"
server_key_iv: "bcc7f4cc7a2fa26b75c8aa5e1de1a989"
server_key_version: "encryptionkey@0"
I20260811 09:01:09.245317 11809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
format_stamp: "Formatted at 2026-08-11 09:01:09 on dist-test-slave-cbsf"
server_key: "c7f10a9e36ae3e0cf96a920ed3d28ce8"
server_key_iv: "bcc7f4cc7a2fa26b75c8aa5e1de1a989"
server_key_version: "encryptionkey@0"
I20260811 09:01:09.250303 11809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:01:09.253635 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:09.254886 11809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:01:09.255026 11809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
format_stamp: "Formatted at 2026-08-11 09:01:09 on dist-test-slave-cbsf"
server_key: "c7f10a9e36ae3e0cf96a920ed3d28ce8"
server_key_iv: "bcc7f4cc7a2fa26b75c8aa5e1de1a989"
server_key_version: "encryptionkey@0"
I20260811 09:01:09.255199 11809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:09.280113 11809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:09.284461 11809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:09.284822 11809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:09.287326 11809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:09.287473 11809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:09.287604 11809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:09.287701 11809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:09.289093 11649 heartbeater.cc:503] Master 127.9.40.126:36749 was elected leader, sending a full tablet report...
Aug 11 09:01:09 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438869, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:09.313704 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.288114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46115 (local address 127.9.40.126:36749)
0811 09:01:09.288350 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.288359 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.289829 (+ 1470us) server_negotiation.cc:407] Connection header received
0811 09:01:09.292113 (+ 2284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.292122 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.292202 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.292257 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:09.294895 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.296234 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.298784 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.299432 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.299617 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.304499 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:09.304509 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:09.304517 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:09.304582 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:09.308189 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.309334 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.309341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.309347 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.309504 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.310157 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.310161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.310163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.310534 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:09.310695 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.313005 (+ 2310us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.313301 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:01:09.314891 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.288619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.67:46115)
0811 09:01:09.289674 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.289727 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.291745 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.292763 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.292776 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.293721 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:09.294605 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.294637 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.296409 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.296414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.298571 (+ 2157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.298591 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.299718 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.299727 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.299962 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.300897 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:09.300918 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:09.304212 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:09.308435 (+ 4223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.308460 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.308480 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.309161 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.309662 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.309670 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.309674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.309985 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.310753 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:09.310759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:09.311248 (+ 489us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.311528 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.312105 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":511,"threads_started":1}
I20260811 09:01:09.336257 11809 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38535
I20260811 09:01:09.336344 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.321639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60599 (local address 127.9.40.126:36749)
0811 09:01:09.321787 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.321791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.322824 (+ 1033us) server_negotiation.cc:407] Connection header received
0811 09:01:09.323057 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.323061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.323119 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.323227 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:09.324278 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.325061 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.326382 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.326851 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.326988 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.329354 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:09.329359 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:09.329362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:09.329391 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:09.332894 (+ 3503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.333729 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.333735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.333738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.333908 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.334375 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.334380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.334383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.334792 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:09.334985 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.335697 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.335949 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:01:09.336375 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.322096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.9.40.67:60599)
0811 09:01:09.322720 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.322739 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.322896 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.323253 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.323257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.323831 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:09.324130 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.324139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.325201 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.325207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.326236 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.326248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.327032 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.327040 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.327227 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.327903 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:09.327919 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:09.329173 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:09.333023 (+ 3850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.333029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.333033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.333573 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.334084 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.334092 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.334100 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.334265 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.334947 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:09.334954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:09.335202 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.335366 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.335712 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":369,"threads_started":1}
I20260811 09:01:09.336802 11962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38535 every 8 connection(s)
I20260811 09:01:09.337942 11809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:09.340152 11963 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36749
I20260811 09:01:09.340449 11963 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:09.341159 11963 heartbeater.cc:511] Master 127.9.40.126:36749 requested a full tablet report, sending...
I20260811 09:01:09.342335 11416 ts_manager.cc:194] Registered new tserver with Master: 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535)
I20260811 09:01:09.343158 11416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:46115
I20260811 09:01:09.344086 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 11809
I20260811 09:01:09.344256 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:09.344745 9377 external_mini_cluster.cc:1428] Setting key eddb20b41c841426d340b824f9f8a6c2
I20260811 09:01:09.347865 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:01:09 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:01:09 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:09.401099 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.386615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.9.40.126:36749)
0811 09:01:09.386836 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.386844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.387114 (+ 270us) server_negotiation.cc:407] Connection header received
0811 09:01:09.387349 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.387354 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.387439 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.387546 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:09.389083 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.389934 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.391254 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.391700 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.391849 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.395995 (+ 4146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:09.396000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:09.396004 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:09.396037 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:09.398543 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.399086 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.399090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.399092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.399209 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.399653 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.399659 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.399662 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.400023 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:09.400236 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.400473 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.400643 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:01:09.405117 11416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:09.411917 11416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:09.438463 11980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.424627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43123 (local address 127.0.0.1:47086)
0811 09:01:09.427459 (+ 2832us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.427499 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.427686 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.429513 (+ 1827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.429521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.429562 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.431388 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.431409 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.433304 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.433311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.435505 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.435520 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.437353 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.437359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.437572 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.437626 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.437792 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.437990 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2531,"mutex_wait_us":1,"thread_start_us":4088,"threads_started":1}
I20260811 09:01:09.438798 11977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.425683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.0.0.1:54176)
0811 09:01:09.428977 (+ 3294us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.428997 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.429151 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.430286 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.430294 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.430329 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.431158 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.431176 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.433348 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.433352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.435757 (+ 2405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.435771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.437926 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.437936 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.438114 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.438154 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.438248 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.438371 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3168,"mutex_wait_us":65,"thread_start_us":1929,"threads_started":1}
I20260811 09:01:09.439237 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.424862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47086 (local address 127.9.40.66:43123)
0811 09:01:09.428613 (+ 3751us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.428619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.428643 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:01:09.428796 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.428803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.428976 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.429241 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.431600 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.433160 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.435661 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.437115 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.437288 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.437495 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.438733 (+ 1238us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.438911 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3627,"thread_start_us":3448,"threads_started":1}
I20260811 09:01:09.439972 11981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.429150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:34569 (local address 127.0.0.1:59104)
0811 09:01:09.432083 (+ 2933us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.432096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.432231 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.432744 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.432748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.432774 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.433083 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.433091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.434921 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.434927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.436676 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.436742 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.439145 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.439153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.439357 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.439389 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.439521 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.439641 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2829,"thread_start_us":2696,"threads_started":1}
I20260811 09:01:09.440006 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.429480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59104 (local address 127.9.40.65:34569)
0811 09:01:09.431632 (+ 2152us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.431643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.432193 (+ 550us) server_negotiation.cc:407] Connection header received
0811 09:01:09.432390 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.432395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.432548 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.432748 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.433242 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.434777 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.436883 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.438230 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.438354 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.438851 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.439562 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.439710 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1965,"thread_start_us":1363,"threads_started":1}
I20260811 09:01:09.440403 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.425250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54176 (local address 127.9.40.67:38535)
0811 09:01:09.426846 (+ 1596us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.426855 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.429294 (+ 2439us) server_negotiation.cc:407] Connection header received
0811 09:01:09.429415 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.429426 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.429691 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.429896 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.431439 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.433167 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.435926 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.437446 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.437572 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.437773 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.439126 (+ 1353us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.439275 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":1242,"threads_started":1}
I20260811 09:01:09.442741 11725 tablet_service.cc:1467] Processing CreateTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (DEFAULT_TABLE table=test-table [id=f070b386874349a1a39cfa88c42b6976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.444216 11725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.443727 11883 tablet_service.cc:1467] Processing CreateTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (DEFAULT_TABLE table=test-table [id=f070b386874349a1a39cfa88c42b6976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.443863 11571 tablet_service.cc:1467] Processing CreateTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (DEFAULT_TABLE table=test-table [id=f070b386874349a1a39cfa88c42b6976]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.445099 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.445113 11883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.455533 11983 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Bootstrap starting.
I20260811 09:01:09.459192 11984 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Bootstrap starting.
I20260811 09:01:09.460244 11983 tablet_bootstrap.cc:654] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.460862 11985 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Bootstrap starting.
I20260811 09:01:09.461726 11983 log.cc:824] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:09.462813 11984 tablet_bootstrap.cc:654] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.463785 11985 tablet_bootstrap.cc:654] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.464025 11984 log.cc:824] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:09.464895 11985 log.cc:824] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:09.465015 11983 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: No bootstrap required, opened a new log
I20260811 09:01:09.465329 11983 ts_tablet_manager.cc:1397] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 09:01:09.467778 11984 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: No bootstrap required, opened a new log
I20260811 09:01:09.468032 11984 ts_tablet_manager.cc:1397] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260811 09:01:09.469856 11985 tablet_bootstrap.cc:492] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: No bootstrap required, opened a new log
I20260811 09:01:09.470125 11985 ts_tablet_manager.cc:1397] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260811 09:01:09.475878 11983 raft_consensus.cc:348] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.476307 11983 raft_consensus.cc:374] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.476404 11983 raft_consensus.cc:729] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.477100 11985 raft_consensus.cc:348] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.477092 11983 consensus_queue.cc:260] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.477461 11985 raft_consensus.cc:374] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.477586 11985 raft_consensus.cc:729] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.478318 11804 heartbeater.cc:503] Master 127.9.40.126:36749 was elected leader, sending a full tablet report...
I20260811 09:01:09.478266 11985 consensus_queue.cc:260] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.478971 11983 ts_tablet_manager.cc:1428] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 09:01:09.479199 11984 raft_consensus.cc:348] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.479705 11984 raft_consensus.cc:374] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.479826 11984 raft_consensus.cc:729] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.480078 11963 heartbeater.cc:503] Master 127.9.40.126:36749 was elected leader, sending a full tablet report...
I20260811 09:01:09.480470 11985 ts_tablet_manager.cc:1428] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260811 09:01:09.480520 11984 consensus_queue.cc:260] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.482594 11984 ts_tablet_manager.cc:1428] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260811 09:01:09.531371 11650 tablet.cc:2369] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:09.546494 11989 raft_consensus.cc:482] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:09.546850 11989 raft_consensus.cc:504] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.548683 11989 leader_election.cc:288] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535), 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569)
I20260811 09:01:09.556061 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.549255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44557 (local address 127.9.40.65:34569)
0811 09:01:09.549444 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.549450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.550171 (+ 721us) server_negotiation.cc:407] Connection header received
0811 09:01:09.550387 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.550393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.550490 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.550590 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.551301 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.552200 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.554254 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.555023 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.555175 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.555301 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.555634 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.555765 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:01:09.556077 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.549146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43199 (local address 127.9.40.67:38535)
0811 09:01:09.549360 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.549366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.550607 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 09:01:09.550766 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.550772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.550854 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.550951 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.551614 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.552347 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.554285 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.555034 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.555177 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.555297 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.555679 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.555805 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 09:01:09.556231 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.549546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.9.40.66:43199)
0811 09:01:09.550480 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.550498 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.550623 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.550973 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.550976 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.550998 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.551396 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.551410 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.552490 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.552494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.554107 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.554119 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.555329 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.555338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.555496 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.555519 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.555634 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.555764 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":62,"thread_start_us":550,"threads_started":1}
I20260811 09:01:09.556594 11992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.549118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:34569 (local address 127.9.40.66:44557)
0811 09:01:09.549994 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.550019 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.550213 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.550625 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.550632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.550667 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.551135 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.551149 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.552333 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.552339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.554093 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.554106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.555291 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.555297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.555454 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.555498 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.555573 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.555723 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"mutex_wait_us":95,"thread_start_us":494,"threads_started":1}
I20260811 09:01:09.557536 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac9d6cfbbf56465eaca35b5c75e5cbb1" candidate_uuid: "62fc6e4a85f7474bb74fb0287c30a897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" is_pre_election: true
I20260811 09:01:09.557746 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac9d6cfbbf56465eaca35b5c75e5cbb1" candidate_uuid: "62fc6e4a85f7474bb74fb0287c30a897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf" is_pre_election: true
I20260811 09:01:09.558127 11909 raft_consensus.cc:2454] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fc6e4a85f7474bb74fb0287c30a897 in term 0.
I20260811 09:01:09.558251 11600 raft_consensus.cc:2454] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62fc6e4a85f7474bb74fb0287c30a897 in term 0.
I20260811 09:01:09.558876 11672 leader_election.cc:302] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 62fc6e4a85f7474bb74fb0287c30a897; no voters:
I20260811 09:01:09.559301 11989 raft_consensus.cc:2781] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:09.559433 11989 raft_consensus.cc:482] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:09.559567 11989 raft_consensus.cc:3037] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.561741 11989 raft_consensus.cc:504] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.562305 11989 leader_election.cc:288] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [CANDIDATE]: Term 1 election: Requested vote from peers 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535), 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569)
I20260811 09:01:09.562736 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac9d6cfbbf56465eaca35b5c75e5cbb1" candidate_uuid: "62fc6e4a85f7474bb74fb0287c30a897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
I20260811 09:01:09.562762 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac9d6cfbbf56465eaca35b5c75e5cbb1" candidate_uuid: "62fc6e4a85f7474bb74fb0287c30a897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf"
I20260811 09:01:09.562947 11909 raft_consensus.cc:3037] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.562947 11600 raft_consensus.cc:3037] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.566156 11600 raft_consensus.cc:2454] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc6e4a85f7474bb74fb0287c30a897 in term 1.
I20260811 09:01:09.566156 11909 raft_consensus.cc:2454] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62fc6e4a85f7474bb74fb0287c30a897 in term 1.
I20260811 09:01:09.566552 11670 leader_election.cc:302] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62fc6e4a85f7474bb74fb0287c30a897, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:09.566867 11989 raft_consensus.cc:2781] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:09.567276 11989 raft_consensus.cc:686] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 LEADER]: Becoming Leader. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Running, Role: LEADER
I20260811 09:01:09.567765 11989 consensus_queue.cc:237] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } }
I20260811 09:01:09.572594 11414 catalog_manager.cc:5495] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 reported cstate change: term changed from 0 to 1, leader changed from <none> to 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "62fc6e4a85f7474bb74fb0287c30a897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } health_report { overall_health: UNKNOWN } } }
W20260811 09:01:09.577080 11805 tablet.cc:2369] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:09.589919 11964 tablet.cc:2369] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:01:09 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438867, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:09.620388 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.607070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36749 (local address 127.0.0.1:40672)
0811 09:01:09.607658 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.607676 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.607793 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.608160 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.608163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.608910 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:09.609283 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.609296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.610282 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.610285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.611312 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.611323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.612030 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.612035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.612143 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.612844 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:09.612861 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:09.615310 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:09.618074 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.618101 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.618111 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.618543 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.618924 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:09.618928 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:09.618930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:09.619072 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:09.619525 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:09.619531 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:09.619676 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.619800 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.619986 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":329,"threads_started":1}
I20260811 09:01:09.620391 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.607044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40672 (local address 127.9.40.126:36749)
0811 09:01:09.607211 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.607216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.607767 (+ 551us) server_negotiation.cc:407] Connection header received
0811 09:01:09.607905 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.607909 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.608018 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.608096 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:09.609417 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.610158 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.611440 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.611843 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.611943 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.615482 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:09.615486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:09.615490 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:09.615517 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:09.617951 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.618675 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.618679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.618681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.618806 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:09.619166 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:09.619170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:09.619171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:09.619409 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:09.619534 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.619881 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.620124 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":63}
I20260811 09:01:09.626204 11414 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40672:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:01:09.627478 11414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:09.634217 11571 tablet_service.cc:1467] Processing CreateTablet for tablet af5ae8a9701f4964b42119762a83724c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=519a0ad47fde4b8aa277d644245d62c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:09.634299 11725 tablet_service.cc:1467] Processing CreateTablet for tablet af5ae8a9701f4964b42119762a83724c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=519a0ad47fde4b8aa277d644245d62c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:09.634563 11883 tablet_service.cc:1467] Processing CreateTablet for tablet af5ae8a9701f4964b42119762a83724c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=519a0ad47fde4b8aa277d644245d62c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:01:09.635100 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af5ae8a9701f4964b42119762a83724c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.635214 11725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af5ae8a9701f4964b42119762a83724c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.635339 11883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af5ae8a9701f4964b42119762a83724c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.640187 11984 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: Bootstrap starting.
I20260811 09:01:09.641680 11984 tablet_bootstrap.cc:654] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.642509 11985 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: Bootstrap starting.
I20260811 09:01:09.644629 11985 tablet_bootstrap.cc:654] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.645274 11983 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: Bootstrap starting.
I20260811 09:01:09.647379 11983 tablet_bootstrap.cc:654] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.649420 11984 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: No bootstrap required, opened a new log
I20260811 09:01:09.649627 11984 ts_tablet_manager.cc:1397] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 09:01:09.650625 11984 raft_consensus.cc:348] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.650949 11984 raft_consensus.cc:374] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.651068 11984 raft_consensus.cc:729] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.652593 11984 consensus_queue.cc:260] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.653471 11984 ts_tablet_manager.cc:1428] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:09.653268 11991 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.653689 11991 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.654999 11983 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: No bootstrap required, opened a new log
I20260811 09:01:09.655534 11985 tablet_bootstrap.cc:492] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: No bootstrap required, opened a new log
I20260811 09:01:09.655702 11985 ts_tablet_manager.cc:1397] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20260811 09:01:09.655932 11983 ts_tablet_manager.cc:1397] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260811 09:01:09.656503 11985 raft_consensus.cc:348] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.656852 11985 raft_consensus.cc:374] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.656810 11983 raft_consensus.cc:348] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.656980 11985 raft_consensus.cc:729] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.657065 11983 raft_consensus.cc:374] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.657331 11983 raft_consensus.cc:729] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.657317 11985 consensus_queue.cc:260] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.657605 11983 consensus_queue.cc:260] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.657778 11990 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.658159 11990 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.658034 11989 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.658228 11983 ts_tablet_manager.cc:1428] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:01:09.658320 11989 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.658631 11985 ts_tablet_manager.cc:1428] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:09.670974 11990 raft_consensus.cc:482] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:09.671191 11990 raft_consensus.cc:504] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.672989 11990 leader_election.cc:288] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569), 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123)
I20260811 09:01:09.680193 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.673279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49879 (local address 127.9.40.65:34569)
0811 09:01:09.673479 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.673487 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.673719 (+ 232us) server_negotiation.cc:407] Connection header received
0811 09:01:09.673794 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.673800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.673901 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.673994 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.674560 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.675665 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.678037 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.679260 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.679382 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.679487 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.679784 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.679913 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:01:09.680238 11836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.673230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:34569 (local address 127.9.40.67:49879)
0811 09:01:09.673451 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.673471 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.673605 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.674151 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.674156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.674186 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.674406 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.674415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.675798 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.675805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.677856 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.677872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.679422 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.679428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.679611 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.679650 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.679751 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.679894 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":1}
I20260811 09:01:09.680922 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.674038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46161 (local address 127.9.40.66:43123)
0811 09:01:09.674251 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.674258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.674628 (+ 370us) server_negotiation.cc:407] Connection header received
0811 09:01:09.674896 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.674901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.674982 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.675049 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:09.675705 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.676789 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.678824 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.679778 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.679890 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.679986 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.680549 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.680700 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:09.680876 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af5ae8a9701f4964b42119762a83724c" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf" is_pre_election: true
I20260811 09:01:09.681121 11600 raft_consensus.cc:2454] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 0.
I20260811 09:01:09.681614 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.673631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43123 (local address 127.9.40.67:46161)
0811 09:01:09.674523 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.674543 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.674730 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.675213 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.675216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.675235 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:09.675529 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.675536 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.676984 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.676989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.678645 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.678657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.680134 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.680140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.680347 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.680386 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.681107 (+ 721us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.681239 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"mutex_wait_us":1,"thread_start_us":626,"threads_started":1}
I20260811 09:01:09.681565 11827 leader_election.cc:302] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:09.682001 11990 raft_consensus.cc:2781] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:09.682125 11990 raft_consensus.cc:482] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:09.682199 11990 raft_consensus.cc:3037] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.682742 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af5ae8a9701f4964b42119762a83724c" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897" is_pre_election: true
I20260811 09:01:09.683244 11753 raft_consensus.cc:2454] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 0.
I20260811 09:01:09.684911 11990 raft_consensus.cc:504] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.685457 11990 leader_election.cc:288] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 election: Requested vote from peers 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569), 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123)
I20260811 09:01:09.685973 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af5ae8a9701f4964b42119762a83724c" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf"
I20260811 09:01:09.686069 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af5ae8a9701f4964b42119762a83724c" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897"
I20260811 09:01:09.686224 11600 raft_consensus.cc:3037] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.686290 11753 raft_consensus.cc:3037] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:09.688264 11600 raft_consensus.cc:2454] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 1.
I20260811 09:01:09.688539 11753 raft_consensus.cc:2454] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 1.
I20260811 09:01:09.688673 11827 leader_election.cc:302] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:09.689047 11990 raft_consensus.cc:2781] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:09.689518 11990 raft_consensus.cc:686] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 LEADER]: Becoming Leader. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Running, Role: LEADER
I20260811 09:01:09.690003 11990 consensus_queue.cc:237] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.693331 12008 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: TxnStatusTablet state changed. Reason: New leader 266b08e6fc0e43db8f24fdfd21c8fa29. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.693755 12008 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:09.694401 12011 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:01:09.694700 12011 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:01:09.694855 11416 catalog_manager.cc:5495] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260811 09:01:09.720781 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.715117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.9.40.67:38535)
0811 09:01:09.715299 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.715306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.715330 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:01:09.715541 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.715546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.715668 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.715816 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:09.716196 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.717151 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.718510 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.718983 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.719127 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.719609 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.720101 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.720248 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.720336 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.720419 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:01:09.720767 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.714910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.0.0.1:54192)
0811 09:01:09.715157 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.715181 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.715321 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.715835 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.715841 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.715853 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:09.716063 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.716072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.717278 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.717283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.718363 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.718375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.719162 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.719168 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.719348 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.719419 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.720211 (+ 792us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.720233 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.720314 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.720455 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:01:09.728863 11600 raft_consensus.cc:1261] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Refusing update from remote peer 266b08e6fc0e43db8f24fdfd21c8fa29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:09.728965 11753 raft_consensus.cc:1261] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Refusing update from remote peer 266b08e6fc0e43db8f24fdfd21c8fa29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:09.729916 12008 consensus_queue.cc:1035] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:09.730294 11990 consensus_queue.cc:1035] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:09.741205 11989 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: TxnStatusTablet state changed. Reason: New leader 266b08e6fc0e43db8f24fdfd21c8fa29. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.741580 11989 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.744640 12016 mvcc.cc:205] Tried to move back new op lower bound from 7317253610398789632 to 7317253610254110720. Current Snapshot: MvccSnapshot[applied={T|T < 7317253610398789632}]
I20260811 09:01:09.746609 11991 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: TxnStatusTablet state changed. Reason: New leader 266b08e6fc0e43db8f24fdfd21c8fa29. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.746984 11991 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.749821 12008 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.750090 12008 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:09.754830 12008 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.755101 12008 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 266b08e6fc0e43db8f24fdfd21c8fa29: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:01:09.755877 12018 mvcc.cc:205] Tried to move back new op lower bound from 7317253610398789632 to 7317253610254110720. Current Snapshot: MvccSnapshot[applied={T|T < 7317253610398789632}]
I20260811 09:01:09.758074 11989 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.758466 11989 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 62fc6e4a85f7474bb74fb0287c30a897: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.762185 11991 tablet_replica.cc:439] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } } }
I20260811 09:01:09.762420 11991 tablet_replica.cc:442] T af5ae8a9701f4964b42119762a83724c P 8df479a596ef474e811edf9a387e8ebf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:01:09.770154 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.764641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.9.40.66:43123)
0811 09:01:09.764836 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.764845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.765076 (+ 231us) server_negotiation.cc:407] Connection header received
0811 09:01:09.765303 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.765308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.765404 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.765546 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:09.765896 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.766636 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.767972 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.768359 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.768549 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.769056 (+ 507us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.769538 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.769691 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.769773 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.769864 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:01:09.787235 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.781095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:54019 (local address 127.9.40.67:38535)
0811 09:01:09.781329 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.781338 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.781707 (+ 369us) server_negotiation.cc:407] Connection header received
0811 09:01:09.781936 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.781946 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.782033 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.782135 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:09.782616 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.783622 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.785382 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.785839 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.786004 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.786477 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.786579 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.786679 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.786825 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.786912 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:01:09.787334 12027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.780972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.9.40.66:54019)
0811 09:01:09.781587 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.781608 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.781784 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.782158 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.782162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.782174 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:09.782468 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.782476 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.783747 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.783754 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.785233 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.785246 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.786050 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.786059 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.786206 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.786257 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.786683 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.786698 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.786802 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.786933 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":374,"threads_started":1}
I20260811 09:01:09.796876 11600 raft_consensus.cc:1261] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Refusing update from remote peer 62fc6e4a85f7474bb74fb0287c30a897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:09.797225 11909 raft_consensus.cc:1261] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Refusing update from remote peer 62fc6e4a85f7474bb74fb0287c30a897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:09.797612 11994 consensus_queue.cc:1035] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:09.798014 11989 consensus_queue.cc:1035] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:09.834971 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.827913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43123 (local address 127.9.40.67:34419)
0811 09:01:09.828147 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:09.828170 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:09.828300 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:09.828920 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:09.828925 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:09.828948 (+ 23us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:09.829134 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.829167 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.830720 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.830726 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.832483 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:09.832498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.833574 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.833579 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:09.833760 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.833848 (+ 88us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.834373 (+ 525us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.834383 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:01:09.834459 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 09:01:09.834617 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 09:01:09.835552 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:09.828324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34419 (local address 127.9.40.66:43123)
0811 09:01:09.828487 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:09.828493 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:09.828510 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:01:09.828586 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:09.828591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:09.828663 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:09.828770 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:09.829349 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.830534 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.832652 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:09.833330 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:09.833444 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:09.834126 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:09.834258 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:09.834328 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:09.835231 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 09:01:09.835308 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:01:09.875147 11415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:09.875478 11415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:09.879586 11415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Sending DeleteTablet for 3 replicas of tablet ac9d6cfbbf56465eaca35b5c75e5cbb1
I20260811 09:01:09.880698 11725 tablet_service.cc:1514] Processing DeleteTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:09 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:47086
I20260811 09:01:09.880833 11883 tablet_service.cc:1514] Processing DeleteTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:09 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54176
I20260811 09:01:09.881470 11571 tablet_service.cc:1514] Processing DeleteTablet for tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:09 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59104
I20260811 09:01:09.884140 11415 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:09.885025 11415 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:09.895521 12036 tablet_replica.cc:330] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: stopping tablet replica
I20260811 09:01:09.896140 12037 tablet_replica.cc:330] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: stopping tablet replica
I20260811 09:01:09.896803 11723 tablet_service.cc:1467] Processing CreateTablet for tablet 5720b445ba584c04ba2d73de0263024f (DEFAULT_TABLE table=test-table [id=5421793305784361a271a7c33e2f8bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.896912 11570 tablet_service.cc:1467] Processing CreateTablet for tablet 5720b445ba584c04ba2d73de0263024f (DEFAULT_TABLE table=test-table [id=5421793305784361a271a7c33e2f8bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.897269 12037 raft_consensus.cc:2229] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:09.897434 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5720b445ba584c04ba2d73de0263024f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.897593 11570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5720b445ba584c04ba2d73de0263024f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.898017 12037 raft_consensus.cc:2258] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:09.897575 12036 raft_consensus.cc:2229] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:09.898923 12036 raft_consensus.cc:2258] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:09.901487 12037 ts_tablet_manager.cc:1905] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:09.902961 12036 ts_tablet_manager.cc:1905] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:09.904489 11984 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Bootstrap starting.
I20260811 09:01:09.905205 11883 tablet_service.cc:1467] Processing CreateTablet for tablet 5720b445ba584c04ba2d73de0263024f (DEFAULT_TABLE table=test-table [id=5421793305784361a271a7c33e2f8bb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:09.905656 11883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5720b445ba584c04ba2d73de0263024f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:09.906205 11984 tablet_bootstrap.cc:654] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.909787 11983 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Bootstrap starting.
I20260811 09:01:09.911448 11983 tablet_bootstrap.cc:654] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.912792 11985 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Bootstrap starting.
I20260811 09:01:09.915133 11985 tablet_bootstrap.cc:654] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:09.915831 11984 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: No bootstrap required, opened a new log
I20260811 09:01:09.916015 11984 ts_tablet_manager.cc:1397] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.001s
I20260811 09:01:09.916774 11984 raft_consensus.cc:348] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.917084 11984 raft_consensus.cc:374] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.917214 11984 raft_consensus.cc:729] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.917538 11984 consensus_queue.cc:260] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.920537 11984 ts_tablet_manager.cc:1428] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:09.928113 12036 ts_tablet_manager.cc:1918] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:09.931479 11985 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: No bootstrap required, opened a new log
I20260811 09:01:09.932890 11985 ts_tablet_manager.cc:1397] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.004s
I20260811 09:01:09.933754 11985 raft_consensus.cc:348] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.934041 11985 raft_consensus.cc:374] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.934206 12037 ts_tablet_manager.cc:1918] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:09.934396 12037 log.cc:1196] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/ac9d6cfbbf56465eaca35b5c75e5cbb1
I20260811 09:01:09.934406 12038 tablet_replica.cc:330] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: stopping tablet replica
I20260811 09:01:09.933301 11983 tablet_bootstrap.cc:492] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: No bootstrap required, opened a new log
I20260811 09:01:09.934669 11983 ts_tablet_manager.cc:1397] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.008s
I20260811 09:01:09.934894 12038 raft_consensus.cc:2229] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:09.935352 12038 raft_consensus.cc:2258] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:09.935451 11983 raft_consensus.cc:348] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.935711 11983 raft_consensus.cc:374] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:09.935818 11983 raft_consensus.cc:729] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.935935 11985 raft_consensus.cc:729] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Initialized, Role: FOLLOWER
I20260811 09:01:09.936105 11983 consensus_queue.cc:260] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.936553 11985 consensus_queue.cc:260] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:09.937345 11985 ts_tablet_manager.cc:1428] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:01:09.941249 11983 ts_tablet_manager.cc:1428] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 09:01:09.942025 12036 log.cc:1196] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/ac9d6cfbbf56465eaca35b5c75e5cbb1
I20260811 09:01:09.942677 12036 ts_tablet_manager.cc:1939] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting consensus metadata
I20260811 09:01:09.944520 11400 catalog_manager.cc:4841] TS 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123): tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (table test-table [id=f070b386874349a1a39cfa88c42b6976]) successfully deleted
I20260811 09:01:09.944784 12037 ts_tablet_manager.cc:1939] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting consensus metadata
I20260811 09:01:09.946503 11402 catalog_manager.cc:4841] TS 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535): tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (table test-table [id=f070b386874349a1a39cfa88c42b6976]) successfully deleted
I20260811 09:01:09.948062 12038 ts_tablet_manager.cc:1905] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:09.953527 12038 ts_tablet_manager.cc:1918] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:09.953703 12038 log.cc:1196] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/ac9d6cfbbf56465eaca35b5c75e5cbb1
I20260811 09:01:09.954259 12038 ts_tablet_manager.cc:1939] T ac9d6cfbbf56465eaca35b5c75e5cbb1 P 8df479a596ef474e811edf9a387e8ebf: Deleting consensus metadata
I20260811 09:01:09.955641 11400 catalog_manager.cc:4841] TS 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569): tablet ac9d6cfbbf56465eaca35b5c75e5cbb1 (table test-table [id=f070b386874349a1a39cfa88c42b6976]) successfully deleted
I20260811 09:01:10.192723 11991 raft_consensus.cc:482] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:10.193022 11991 raft_consensus.cc:504] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:10.195019 11991 leader_election.cc:288] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123), 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535)
I20260811 09:01:10.202369 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.195226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.9.40.65:40889)
0811 09:01:10.196041 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:10.196063 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:10.196228 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:10.196796 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:10.196802 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:10.196829 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:10.197161 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.197173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.198593 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.198599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.200284 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.200296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.201493 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.201499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.201730 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.201765 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:01:10.201867 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:01:10.202008 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":594,"threads_started":1}
I20260811 09:01:10.202371 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.195520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40889 (local address 127.9.40.67:38535)
0811 09:01:10.195731 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:10.195737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:10.196174 (+ 437us) server_negotiation.cc:407] Connection header received
0811 09:01:10.196398 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:10.196406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:10.196500 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:10.196605 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:10.197332 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.198438 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.200453 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.201331 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.201470 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.201582 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:10.201922 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 09:01:10.202084 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:01:10.203361 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5720b445ba584c04ba2d73de0263024f" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" is_pre_election: true
I20260811 09:01:10.203663 11909 raft_consensus.cc:2454] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 0.
I20260811 09:01:10.204229 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.195226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43123 (local address 127.9.40.65:51879)
0811 09:01:10.196755 (+ 1529us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:10.196774 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:10.196925 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:10.197438 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:10.197441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:10.197459 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:10.197802 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.197810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.199202 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.199208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.201573 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.201591 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.203403 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.203409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.203624 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.203653 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:01:10.203744 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 09:01:10.203870 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1372,"mutex_wait_us":524,"thread_start_us":684,"threads_started":1}
I20260811 09:01:10.204257 11978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.195621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51879 (local address 127.9.40.66:43123)
0811 09:01:10.195816 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:10.195823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:10.196850 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 09:01:10.197105 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:10.197113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:10.197208 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:10.197297 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:10.197966 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.199046 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.201751 (+ 2705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.203148 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.203315 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.203448 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:10.203860 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 09:01:10.203995 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 09:01:10.204768 11520 leader_election.cc:302] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:10.205195 11991 raft_consensus.cc:2781] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:10.205320 11991 raft_consensus.cc:482] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:10.205430 11991 raft_consensus.cc:3037] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:10.205554 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5720b445ba584c04ba2d73de0263024f" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897" is_pre_election: true
I20260811 09:01:10.205801 11753 raft_consensus.cc:2454] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 0.
I20260811 09:01:10.208653 11991 raft_consensus.cc:504] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:10.209431 11991 leader_election.cc:288] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 election: Requested vote from peers 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123), 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535)
I20260811 09:01:10.209896 11753 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5720b445ba584c04ba2d73de0263024f" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897"
I20260811 09:01:10.210011 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5720b445ba584c04ba2d73de0263024f" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
I20260811 09:01:10.210124 11753 raft_consensus.cc:3037] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:10.210239 11909 raft_consensus.cc:3037] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:10.213200 11909 raft_consensus.cc:2454] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 1.
I20260811 09:01:10.213274 11753 raft_consensus.cc:2454] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 1.
I20260811 09:01:10.213654 11518 leader_election.cc:302] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62fc6e4a85f7474bb74fb0287c30a897, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:10.213953 11991 raft_consensus.cc:2781] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:10.214448 11991 raft_consensus.cc:686] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 1 LEADER]: Becoming Leader. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Running, Role: LEADER
I20260811 09:01:10.214901 11991 consensus_queue.cc:237] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:10.218840 11414 catalog_manager.cc:5495] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df479a596ef474e811edf9a387e8ebf (127.9.40.65). New cstate: current_term: 1 leader_uuid: "8df479a596ef474e811edf9a387e8ebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260811 09:01:10.301061 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.293658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59106 (local address 127.9.40.65:34569)
0811 09:01:10.294233 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:10.294241 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:10.294264 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:01:10.294358 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:10.294362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:10.294439 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:10.294608 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:10.295142 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.296088 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.297729 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.298350 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.298500 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.299165 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:10.299969 (+ 804us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:10.300131 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:10.300298 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:01:10.300372 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":289,"threads_started":1}
I20260811 09:01:10.319325 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.311966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56405 (local address 127.9.40.67:38535)
0811 09:01:10.312172 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:10.312179 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:10.313401 (+ 1222us) server_negotiation.cc:407] Connection header received
0811 09:01:10.313753 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:10.313759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:10.313876 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:10.313980 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:10.315017 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.315749 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.317370 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.317908 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.318062 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.318571 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:10.318690 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:10.318794 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:10.318935 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 09:01:10.319017 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:01:10.319590 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.311785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.9.40.65:56405)
0811 09:01:10.313291 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:10.313312 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:10.313581 (+ 269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:10.313997 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:10.314001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:10.314012 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:10.314862 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.314875 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.315895 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.315900 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.317181 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.317193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.318100 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.318107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.318266 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.318317 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:10.318799 (+ 482us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:10.318813 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 09:01:10.318913 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:01:10.319044 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1233,"thread_start_us":799,"threads_started":1}
I20260811 09:01:10.328454 11909 raft_consensus.cc:1261] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Refusing update from remote peer 8df479a596ef474e811edf9a387e8ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:10.328594 11753 raft_consensus.cc:1261] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Refusing update from remote peer 8df479a596ef474e811edf9a387e8ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:10.329550 11991 consensus_queue.cc:1035] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:10.329988 12046 consensus_queue.cc:1035] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:10.380756 12052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.375521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56783 (local address 127.9.40.65:34569)
0811 09:01:10.375695 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:10.375701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:10.375720 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:10.375788 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:10.375793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:10.375881 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:10.375970 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:10.376594 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.377456 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.378991 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.379415 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.379567 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.379972 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:10.380084 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:10.380181 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:10.380369 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 09:01:10.380461 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:01:10.381446 12060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:10.373877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:34569 (local address 127.9.40.67:56783)
0811 09:01:10.374440 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:10.374461 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:10.374622 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:10.376185 (+ 1563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:10.376190 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:10.376203 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:10.376435 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.376443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.377597 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.377605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.378849 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:10.378862 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:10.379610 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:10.379619 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:10.379774 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:10.379812 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:10.380183 (+ 371us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:10.380198 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 09:01:10.380949 (+ 751us) client_negotiation.cc:240] Negotiation successful
0811 09:01:10.381103 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":317,"threads_started":1}
W20260811 09:01:10.382720 12018 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:01:11.974005 11414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:11.974210 11414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:11.978382 11414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Sending DeleteTablet for 3 replicas of tablet 5720b445ba584c04ba2d73de0263024f
I20260811 09:01:11.979127 11571 tablet_service.cc:1514] Processing DeleteTablet for tablet 5720b445ba584c04ba2d73de0263024f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:11 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59104
I20260811 09:01:11.979235 11723 tablet_service.cc:1514] Processing DeleteTablet for tablet 5720b445ba584c04ba2d73de0263024f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:11 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:47086
I20260811 09:01:11.979228 11883 tablet_service.cc:1514] Processing DeleteTablet for tablet 5720b445ba584c04ba2d73de0263024f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:11 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54176
I20260811 09:01:11.992003 12092 tablet_replica.cc:330] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: stopping tablet replica
I20260811 09:01:11.992457 12092 raft_consensus.cc:2229] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:11.992871 12092 raft_consensus.cc:2258] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:11.994365 12092 ts_tablet_manager.cc:1905] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:11.995872 12090 tablet_replica.cc:330] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: stopping tablet replica
I20260811 09:01:11.996142 12090 raft_consensus.cc:2229] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:11.996457 12091 tablet_replica.cc:330] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: stopping tablet replica
I20260811 09:01:11.996673 12091 raft_consensus.cc:2229] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:11.996811 12090 raft_consensus.cc:2258] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:11.997022 12091 raft_consensus.cc:2258] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:11.998047 12090 ts_tablet_manager.cc:1905] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:11.998383 12091 ts_tablet_manager.cc:1905] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:01:12 dist-test-slave-cbsf krb5kdc[11367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 09:01:12.088024 9377 mini_kdc.cc:317] Time spent kinit for test-user: real 0.108s user 0.001s sys 0.015s
I20260811 09:01:12.092329 12091 ts_tablet_manager.cc:1918] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:12.092522 12091 log.cc:1196] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/5720b445ba584c04ba2d73de0263024f
I20260811 09:01:12.092769 12092 ts_tablet_manager.cc:1918] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:12.092958 12092 log.cc:1196] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/5720b445ba584c04ba2d73de0263024f
I20260811 09:01:12.093075 12091 ts_tablet_manager.cc:1939] T 5720b445ba584c04ba2d73de0263024f P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting consensus metadata
I20260811 09:01:12.093742 12092 ts_tablet_manager.cc:1939] T 5720b445ba584c04ba2d73de0263024f P 62fc6e4a85f7474bb74fb0287c30a897: Deleting consensus metadata
Aug 11 09:01:12 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:12.113276 12104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:12.095773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42980 (local address 127.9.40.126:36749)
0811 09:01:12.096928 (+ 1155us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:12.096936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:12.096965 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:12.097018 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:12.097021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:12.097080 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:12.097265 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:12.098789 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.099632 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.101004 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.101418 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.101704 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:12.106223 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:12.106228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:12.106232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:12.106263 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:12.109819 (+ 3556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:12.110706 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:12.110712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:12.110714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:12.110854 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:12.111320 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:12.111325 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:12.111327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:12.111660 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:12.111928 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:12.112337 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 09:01:12.112619 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":999,"thread_start_us":654,"threads_started":1}
I20260811 09:01:12.117477 11414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:12.118237 11414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:12.125413 11723 tablet_service.cc:1467] Processing CreateTablet for tablet dcfd32e47a55414284fef480d4445667 (DEFAULT_TABLE table=test-table [id=58b5cefad206477bbe70547ead8baa80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.125540 11571 tablet_service.cc:1467] Processing CreateTablet for tablet dcfd32e47a55414284fef480d4445667 (DEFAULT_TABLE table=test-table [id=58b5cefad206477bbe70547ead8baa80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.125622 11883 tablet_service.cc:1467] Processing CreateTablet for tablet dcfd32e47a55414284fef480d4445667 (DEFAULT_TABLE table=test-table [id=58b5cefad206477bbe70547ead8baa80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.125950 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcfd32e47a55414284fef480d4445667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.126098 11883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcfd32e47a55414284fef480d4445667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.131716 12105 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Bootstrap starting.
I20260811 09:01:12.132400 12106 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Bootstrap starting.
I20260811 09:01:12.133756 12105 tablet_bootstrap.cc:654] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.133822 12106 tablet_bootstrap.cc:654] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.136222 12106 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: No bootstrap required, opened a new log
I20260811 09:01:12.136222 12105 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: No bootstrap required, opened a new log
I20260811 09:01:12.136381 12106 ts_tablet_manager.cc:1397] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:12.136370 12105 ts_tablet_manager.cc:1397] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 09:01:12.137277 12105 raft_consensus.cc:348] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.137277 12106 raft_consensus.cc:348] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.137535 12106 raft_consensus.cc:374] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.137535 12105 raft_consensus.cc:374] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.137631 12106 raft_consensus.cc:729] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.137646 12105 raft_consensus.cc:729] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.137907 12105 consensus_queue.cc:260] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.137887 12106 consensus_queue.cc:260] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.138585 12105 ts_tablet_manager.cc:1428] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 09:01:12.139385 12106 ts_tablet_manager.cc:1428] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:12.177054 11402 catalog_manager.cc:4841] TS 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535): tablet 5720b445ba584c04ba2d73de0263024f (table test-table [id=5421793305784361a271a7c33e2f8bb7]) successfully deleted
I20260811 09:01:12.183720 11400 catalog_manager.cc:4841] TS 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123): tablet 5720b445ba584c04ba2d73de0263024f (table test-table [id=5421793305784361a271a7c33e2f8bb7]) successfully deleted
I20260811 09:01:12.191987 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcfd32e47a55414284fef480d4445667. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.192102 12090 ts_tablet_manager.cc:1918] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:12.192294 12090 log.cc:1196] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/5720b445ba584c04ba2d73de0263024f
I20260811 09:01:12.193162 12090 ts_tablet_manager.cc:1939] T 5720b445ba584c04ba2d73de0263024f P 8df479a596ef474e811edf9a387e8ebf: Deleting consensus metadata
I20260811 09:01:12.194470 11400 catalog_manager.cc:4841] TS 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569): tablet 5720b445ba584c04ba2d73de0263024f (table test-table [id=5421793305784361a271a7c33e2f8bb7]) successfully deleted
I20260811 09:01:12.198679 12111 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Bootstrap starting.
I20260811 09:01:12.200240 12111 tablet_bootstrap.cc:654] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.202330 12111 tablet_bootstrap.cc:492] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: No bootstrap required, opened a new log
I20260811 09:01:12.202445 12111 ts_tablet_manager.cc:1397] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 09:01:12.203009 12111 raft_consensus.cc:348] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.203233 12111 raft_consensus.cc:374] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.203312 12111 raft_consensus.cc:729] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.203538 12111 consensus_queue.cc:260] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.204067 12111 ts_tablet_manager.cc:1428] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 09:01:12.577708 12081 raft_consensus.cc:482] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:12.577893 12081 raft_consensus.cc:504] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.578503 12081 leader_election.cc:288] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123), 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535)
I20260811 09:01:12.579241 11752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcfd32e47a55414284fef480d4445667" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897" is_pre_election: true
I20260811 09:01:12.579282 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcfd32e47a55414284fef480d4445667" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" is_pre_election: true
I20260811 09:01:12.579524 11909 raft_consensus.cc:2454] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 0.
I20260811 09:01:12.579526 11752 raft_consensus.cc:2454] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 0.
I20260811 09:01:12.580039 11520 leader_election.cc:302] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:12.580349 12081 raft_consensus.cc:2781] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:12.580516 12081 raft_consensus.cc:482] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:12.580643 12081 raft_consensus.cc:3037] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.583217 12081 raft_consensus.cc:504] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.583835 12081 leader_election.cc:288] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 election: Requested vote from peers 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123), 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535)
I20260811 09:01:12.584300 11909 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcfd32e47a55414284fef480d4445667" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29"
I20260811 09:01:12.584300 11752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcfd32e47a55414284fef480d4445667" candidate_uuid: "8df479a596ef474e811edf9a387e8ebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897"
I20260811 09:01:12.584513 11909 raft_consensus.cc:3037] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.584515 11752 raft_consensus.cc:3037] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.587544 11909 raft_consensus.cc:2454] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 1.
I20260811 09:01:12.587512 11752 raft_consensus.cc:2454] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8df479a596ef474e811edf9a387e8ebf in term 1.
I20260811 09:01:12.587982 11520 leader_election.cc:302] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:12.588306 12081 raft_consensus.cc:2781] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:12.588778 12081 raft_consensus.cc:686] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 1 LEADER]: Becoming Leader. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Running, Role: LEADER
I20260811 09:01:12.589145 12081 consensus_queue.cc:237] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.591840 11415 catalog_manager.cc:5495] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 8df479a596ef474e811edf9a387e8ebf (127.9.40.65). New cstate: current_term: 1 leader_uuid: "8df479a596ef474e811edf9a387e8ebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } health_report { overall_health: UNKNOWN } } }
I20260811 09:01:12.715258 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:12.707112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32790 (local address 127.9.40.65:34569)
0811 09:01:12.707637 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:12.707646 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:12.710680 (+ 3034us) server_negotiation.cc:407] Connection header received
0811 09:01:12.710824 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:12.710828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:12.710890 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:12.710983 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:12.711428 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.712120 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.713603 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.714040 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.714140 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:12.714530 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:12.714657 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:12.714727 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:12.714860 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 09:01:12.714929 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
I20260811 09:01:12.724913 11909 raft_consensus.cc:1261] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Refusing update from remote peer 8df479a596ef474e811edf9a387e8ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:12.724968 11752 raft_consensus.cc:1261] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Refusing update from remote peer 8df479a596ef474e811edf9a387e8ebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:12.725550 12113 consensus_queue.cc:1035] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:12.725836 12081 consensus_queue.cc:1035] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:12.780337 11414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:12.780644 11414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:12.783528 11414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Sending DeleteTablet for 3 replicas of tablet dcfd32e47a55414284fef480d4445667
I20260811 09:01:12.784394 11571 tablet_service.cc:1514] Processing DeleteTablet for tablet dcfd32e47a55414284fef480d4445667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59104
I20260811 09:01:12.784488 11883 tablet_service.cc:1514] Processing DeleteTablet for tablet dcfd32e47a55414284fef480d4445667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54176
I20260811 09:01:12.784569 11723 tablet_service.cc:1514] Processing DeleteTablet for tablet dcfd32e47a55414284fef480d4445667 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:47086
I20260811 09:01:12.785274 12135 tablet_replica.cc:330] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: stopping tablet replica
I20260811 09:01:12.785477 12137 tablet_replica.cc:330] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: stopping tablet replica
I20260811 09:01:12.785530 12135 raft_consensus.cc:2229] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:12.785696 12137 raft_consensus.cc:2229] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:12.785969 12137 raft_consensus.cc:2258] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:12.786156 12135 raft_consensus.cc:2258] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:12.786484 11414 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 09:01:12.787138 12137 ts_tablet_manager.cc:1905] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 09:01:12.787358 11414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:12.787384 12135 ts_tablet_manager.cc:1905] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:12.790267 12136 tablet_replica.cc:330] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: stopping tablet replica
I20260811 09:01:12.790478 12136 raft_consensus.cc:2229] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:12.790746 12136 raft_consensus.cc:2258] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:12.793546 12136 ts_tablet_manager.cc:1905] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:12.794914 12135 ts_tablet_manager.cc:1918] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:12.794977 12137 ts_tablet_manager.cc:1918] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:12.795148 12137 log.cc:1196] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/dcfd32e47a55414284fef480d4445667
I20260811 09:01:12.795171 12135 log.cc:1196] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/dcfd32e47a55414284fef480d4445667
I20260811 09:01:12.795853 12135 ts_tablet_manager.cc:1939] T dcfd32e47a55414284fef480d4445667 P 8df479a596ef474e811edf9a387e8ebf: Deleting consensus metadata
I20260811 09:01:12.796213 11571 tablet_service.cc:1467] Processing CreateTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa (DEFAULT_TABLE table=test-table [id=75c3ec74e5dd4e508d340e636747138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.796383 11723 tablet_service.cc:1467] Processing CreateTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa (DEFAULT_TABLE table=test-table [id=75c3ec74e5dd4e508d340e636747138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.796633 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4810b0f1da9d41d5a45f9815ed90cdaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.796814 11723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4810b0f1da9d41d5a45f9815ed90cdaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.800304 12136 ts_tablet_manager.cc:1918] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:01:12.800448 12136 log.cc:1196] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/dcfd32e47a55414284fef480d4445667
I20260811 09:01:12.800944 12136 ts_tablet_manager.cc:1939] T dcfd32e47a55414284fef480d4445667 P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting consensus metadata
I20260811 09:01:12.802172 11400 catalog_manager.cc:4841] TS 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569): tablet dcfd32e47a55414284fef480d4445667 (table test-table [id=58b5cefad206477bbe70547ead8baa80]) successfully deleted
I20260811 09:01:12.802389 11402 catalog_manager.cc:4841] TS 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535): tablet dcfd32e47a55414284fef480d4445667 (table test-table [id=58b5cefad206477bbe70547ead8baa80]) successfully deleted
I20260811 09:01:12.795786 12137 ts_tablet_manager.cc:1939] T dcfd32e47a55414284fef480d4445667 P 62fc6e4a85f7474bb74fb0287c30a897: Deleting consensus metadata
I20260811 09:01:12.803329 11883 tablet_service.cc:1467] Processing CreateTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa (DEFAULT_TABLE table=test-table [id=75c3ec74e5dd4e508d340e636747138b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:12.804018 11400 catalog_manager.cc:4841] TS 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123): tablet dcfd32e47a55414284fef480d4445667 (table test-table [id=58b5cefad206477bbe70547ead8baa80]) successfully deleted
I20260811 09:01:12.804513 11883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4810b0f1da9d41d5a45f9815ed90cdaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:12.805299 12139 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Bootstrap starting.
I20260811 09:01:12.807299 12139 tablet_bootstrap.cc:654] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.809450 12138 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Bootstrap starting.
I20260811 09:01:12.810767 12139 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: No bootstrap required, opened a new log
I20260811 09:01:12.810983 12139 ts_tablet_manager.cc:1397] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 09:01:12.811900 12139 raft_consensus.cc:348] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.812146 12138 tablet_bootstrap.cc:654] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.812201 12139 raft_consensus.cc:374] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.812304 12139 raft_consensus.cc:729] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62fc6e4a85f7474bb74fb0287c30a897, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.812579 12139 consensus_queue.cc:260] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.813329 12141 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Bootstrap starting.
I20260811 09:01:12.813678 12139 ts_tablet_manager.cc:1428] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:01:12.815521 12138 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: No bootstrap required, opened a new log
I20260811 09:01:12.815699 12138 ts_tablet_manager.cc:1397] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 09:01:12.816654 12141 tablet_bootstrap.cc:654] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:12.816485 12138 raft_consensus.cc:348] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.816787 12138 raft_consensus.cc:374] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.816883 12138 raft_consensus.cc:729] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df479a596ef474e811edf9a387e8ebf, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.817157 12138 consensus_queue.cc:260] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.817912 12138 ts_tablet_manager.cc:1428] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:01:12.818975 12141 tablet_bootstrap.cc:492] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: No bootstrap required, opened a new log
I20260811 09:01:12.819136 12141 ts_tablet_manager.cc:1397] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260811 09:01:12.819914 12141 raft_consensus.cc:348] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.820184 12141 raft_consensus.cc:374] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:12.820320 12141 raft_consensus.cc:729] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Initialized, Role: FOLLOWER
I20260811 09:01:12.820583 12141 consensus_queue.cc:260] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.821141 12141 ts_tablet_manager.cc:1428] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:01:12.838662 12116 raft_consensus.cc:482] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:12.838881 12116 raft_consensus.cc:504] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.839545 12116 leader_election.cc:288] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569), 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123)
I20260811 09:01:12.840080 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4810b0f1da9d41d5a45f9815ed90cdaa" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf" is_pre_election: true
I20260811 09:01:12.840185 11752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4810b0f1da9d41d5a45f9815ed90cdaa" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897" is_pre_election: true
I20260811 09:01:12.840354 11600 raft_consensus.cc:2454] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 0.
I20260811 09:01:12.840466 11752 raft_consensus.cc:2454] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 0.
I20260811 09:01:12.840945 11827 leader_election.cc:302] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 62fc6e4a85f7474bb74fb0287c30a897; no voters:
I20260811 09:01:12.841244 12116 raft_consensus.cc:2781] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:12.841370 12116 raft_consensus.cc:482] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:12.841492 12116 raft_consensus.cc:3037] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.843575 12116 raft_consensus.cc:504] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.844159 12116 leader_election.cc:288] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 election: Requested vote from peers 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569), 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123)
I20260811 09:01:12.844583 11600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4810b0f1da9d41d5a45f9815ed90cdaa" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8df479a596ef474e811edf9a387e8ebf"
I20260811 09:01:12.844739 11752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4810b0f1da9d41d5a45f9815ed90cdaa" candidate_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "62fc6e4a85f7474bb74fb0287c30a897"
I20260811 09:01:12.844822 11600 raft_consensus.cc:3037] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.844945 11752 raft_consensus.cc:3037] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:12.847344 11600 raft_consensus.cc:2454] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 1.
I20260811 09:01:12.847880 11827 leader_election.cc:302] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 266b08e6fc0e43db8f24fdfd21c8fa29, 8df479a596ef474e811edf9a387e8ebf; no voters:
I20260811 09:01:12.848146 11752 raft_consensus.cc:2454] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 266b08e6fc0e43db8f24fdfd21c8fa29 in term 1.
I20260811 09:01:12.848306 12116 raft_consensus.cc:2781] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:12.848453 12116 raft_consensus.cc:686] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 LEADER]: Becoming Leader. State: Replica: 266b08e6fc0e43db8f24fdfd21c8fa29, State: Running, Role: LEADER
I20260811 09:01:12.848769 12116 consensus_queue.cc:237] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } }
I20260811 09:01:12.851394 11415 catalog_manager.cc:5495] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "266b08e6fc0e43db8f24fdfd21c8fa29" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38535 } health_report { overall_health: HEALTHY } } }
I20260811 09:01:12.874812 12145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:12.869871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60126 (local address 127.9.40.67:38535)
0811 09:01:12.870451 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:12.870455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:12.870474 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:12.870521 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:12.870525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:12.870591 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:12.870669 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:12.871034 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.871826 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.873170 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.873600 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.873689 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:12.874077 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:12.874180 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:12.874247 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:12.874368 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 09:01:12.874425 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":349,"threads_started":1}
I20260811 09:01:12.887844 12145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:12.882884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53421 (local address 127.9.40.67:38535)
0811 09:01:12.883094 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:12.883102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:12.883273 (+ 171us) server_negotiation.cc:407] Connection header received
0811 09:01:12.883513 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:12.883520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:12.883591 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:12.883680 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:12.884139 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.884902 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.886178 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.886566 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.886701 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:12.887081 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:12.887167 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:12.887260 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:12.887387 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 09:01:12.887473 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":61}
I20260811 09:01:12.887964 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:12.882599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38535 (local address 127.9.40.67:53421)
0811 09:01:12.883183 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:12.883202 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:12.883342 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:12.883701 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:12.883706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:12.883716 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:01:12.883988 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:12.883995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.885038 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.885043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:12.886040 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:12.886051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:12.886753 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:12.886760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:12.886882 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:12.886918 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:01:12.887259 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:01:12.887272 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:01:12.887370 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:01:12.887510 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"mutex_wait_us":2,"thread_start_us":307,"threads_started":1}
I20260811 09:01:12.895187 11752 raft_consensus.cc:1261] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Refusing update from remote peer 266b08e6fc0e43db8f24fdfd21c8fa29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:12.895231 11600 raft_consensus.cc:1261] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Refusing update from remote peer 266b08e6fc0e43db8f24fdfd21c8fa29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:12.895722 12127 consensus_queue.cc:1035] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "62fc6e4a85f7474bb74fb0287c30a897" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:12.896045 12116 consensus_queue.cc:1035] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8df479a596ef474e811edf9a387e8ebf" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 34569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 09:01:12.916883 12115 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:01:14.521886 11414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:14.522334 11414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42980:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:14.533002 11414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f44589154fa461eb057ef57c77ef37a: Sending DeleteTablet for 3 replicas of tablet 4810b0f1da9d41d5a45f9815ed90cdaa
I20260811 09:01:14.534325 11571 tablet_service.cc:1514] Processing DeleteTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:14 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59104
I20260811 09:01:14.534837 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11500
I20260811 09:01:14.534909 11882 tablet_service.cc:1514] Processing DeleteTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:14 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54176
I20260811 09:01:14.535024 11723 tablet_service.cc:1514] Processing DeleteTablet for tablet 4810b0f1da9d41d5a45f9815ed90cdaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:14 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:47086
I20260811 09:01:14.537339 12177 tablet_replica.cc:330] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: stopping tablet replica
I20260811 09:01:14.537689 12177 raft_consensus.cc:2229] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:14.537988 12177 raft_consensus.cc:2258] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:14.539484 12177 ts_tablet_manager.cc:1905] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:14.547582 12178 tablet_replica.cc:330] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: stopping tablet replica
I20260811 09:01:14.547842 12178 raft_consensus.cc:2229] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:14.548540 12178 raft_consensus.cc:2258] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:14.549954 12178 ts_tablet_manager.cc:1905] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:14.550797 12179 tablet_replica.cc:330] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: stopping tablet replica
I20260811 09:01:14.551075 12179 raft_consensus.cc:2229] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:14.551395 12179 raft_consensus.cc:2258] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:14.552563 12177 ts_tablet_manager.cc:1918] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:14.552995 12177 log.cc:1196] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/4810b0f1da9d41d5a45f9815ed90cdaa
I20260811 09:01:14.553965 12177 ts_tablet_manager.cc:1939] T 4810b0f1da9d41d5a45f9815ed90cdaa P 8df479a596ef474e811edf9a387e8ebf: Deleting consensus metadata
I20260811 09:01:14.555413 11400 catalog_manager.cc:4841] TS 8df479a596ef474e811edf9a387e8ebf (127.9.40.65:34569): tablet 4810b0f1da9d41d5a45f9815ed90cdaa (table test-table [id=75c3ec74e5dd4e508d340e636747138b]) successfully deleted
I20260811 09:01:14.557296 12179 ts_tablet_manager.cc:1905] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:14.560535 12178 ts_tablet_manager.cc:1918] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:14.560817 12178 log.cc:1196] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/4810b0f1da9d41d5a45f9815ed90cdaa
I20260811 09:01:14.561571 12178 ts_tablet_manager.cc:1939] T 4810b0f1da9d41d5a45f9815ed90cdaa P 266b08e6fc0e43db8f24fdfd21c8fa29: Deleting consensus metadata
I20260811 09:01:14.563203 11402 catalog_manager.cc:4841] TS 266b08e6fc0e43db8f24fdfd21c8fa29 (127.9.40.67:38535): tablet 4810b0f1da9d41d5a45f9815ed90cdaa (table test-table [id=75c3ec74e5dd4e508d340e636747138b]) successfully deleted
I20260811 09:01:14.563396 12179 ts_tablet_manager.cc:1918] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:01:14.563570 12179 log.cc:1196] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/4810b0f1da9d41d5a45f9815ed90cdaa
I20260811 09:01:14.564265 12179 ts_tablet_manager.cc:1939] T 4810b0f1da9d41d5a45f9815ed90cdaa P 62fc6e4a85f7474bb74fb0287c30a897: Deleting consensus metadata
I20260811 09:01:14.565532 11400 catalog_manager.cc:4841] TS 62fc6e4a85f7474bb74fb0287c30a897 (127.9.40.66:43123): tablet 4810b0f1da9d41d5a45f9815ed90cdaa (table test-table [id=75c3ec74e5dd4e508d340e636747138b]) successfully deleted
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:14.577092 12180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:14.536356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.9.40.65:34569)
0811 09:01:14.537611 (+ 1255us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:14.537621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:14.539071 (+ 1450us) server_negotiation.cc:407] Connection header received
0811 09:01:14.539152 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:14.539161 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:14.539293 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:14.539419 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:14.542285 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.544065 (+ 1780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.556422 (+ 12357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.557564 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.557702 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:14.569212 (+ 11510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:14.569229 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:14.569247 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:14.569284 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:14.573871 (+ 4587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.574494 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.574501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.574504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.574693 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.575151 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.575154 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.575157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.575577 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:14.575834 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:14.576279 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 09:01:14.576623 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031,"thread_start_us":824,"threads_started":1}
W20260811 09:01:14.577986 11643 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32796
W20260811 09:01:14.578840 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11500: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:14.579118 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 11500
I20260811 09:01:14.607712 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11653
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:14.624348 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:14.608515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.9.40.66:43123)
0811 09:01:14.609183 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:14.609188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:14.609204 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:01:14.609272 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:14.609275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:14.609331 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:14.609421 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:14.611198 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.611882 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.613619 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.614005 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.614219 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:14.618684 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:14.618697 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:14.618713 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:14.618758 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:14.621552 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.622232 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.622239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.622241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.622410 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.622906 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.622909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.622911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.623270 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:14.623503 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:14.623796 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:01:14.623978 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":421,"threads_started":1}
W20260811 09:01:14.625414 11797 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38858
W20260811 09:01:14.626143 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11653: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:14.626366 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 11653
I20260811 09:01:14.654192 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11809
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[11367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438872, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:14.675639 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:14.655075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.9.40.67:38535)
0811 09:01:14.655576 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:14.655581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:14.655596 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:01:14.655649 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:14.655653 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:14.655720 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:14.655838 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:14.657429 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.658530 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.660833 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:14.661481 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:14.661637 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:14.667394 (+ 5757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:14.667418 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:14.667441 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:14.667491 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:14.671978 (+ 4487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.672935 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.672946 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.672950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.673112 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:14.673743 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:14.673748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:14.673751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:14.674197 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:14.674492 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:14.675045 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 09:01:14.675283 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":251,"threads_started":1}
W20260811 09:01:14.676654 11958 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60138
W20260811 09:01:14.677436 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11809: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:14.677611 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 11809
I20260811 09:01:14.715046 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 11380
I20260811 09:01:14.716302 11465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:14.837571 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 11380
2026-08-11T09:01:14Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8976 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[12187](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[12187](info): set up 2 sockets
Aug 11 09:01:14 dist-test-slave-cbsf krb5kdc[12187](info): commencing operation
krb5kdc: starting...
W20260811 09:01:16.025316 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:16 dist-test-slave-cbsf krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:16.240650 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35543
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:16.657061 12202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:16.657465 12202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:16.657625 12202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:16.665339 12202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:16.665486 12202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:16.665572 12202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:16.665654 12202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:16.665733 12202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:16.675336 12202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35543
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:16.676682 12202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:16.678078 12202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:16.690627 12207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:16.690538 12210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:16.690671 12208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:16.691354 12202 server_base.cc:1034] running on GCE node
I20260811 09:01:16.692318 12202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:16.693792 12202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:16.695039 12202 hybrid_clock.cc:648] HybridClock initialized: now 1786438876695005 us; error 53 us; skew 500 ppm
I20260811 09:01:16.695411 12202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:16 dist-test-slave-cbsf krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438876, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:16.698704 12202 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:16.700151 12202 webserver.cc:460] Webserver started at http://127.9.40.126:46457/ using document root <none> and password file <none>
I20260811 09:01:16.700919 12202 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:16.701071 12202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:16.701421 12202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:16.704403 12202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "547fe07a0b3d4c2e86f5045c8637df31"
format_stamp: "Formatted at 2026-08-11 09:01:16 on dist-test-slave-cbsf"
server_key: "af4d971a5ff8be57b8894a0b9c644888"
server_key_iv: "9174d587a762b94fa1a66a0f172e975f"
server_key_version: "encryptionkey@0"
I20260811 09:01:16.705276 12202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "547fe07a0b3d4c2e86f5045c8637df31"
format_stamp: "Formatted at 2026-08-11 09:01:16 on dist-test-slave-cbsf"
server_key: "af4d971a5ff8be57b8894a0b9c644888"
server_key_iv: "9174d587a762b94fa1a66a0f172e975f"
server_key_version: "encryptionkey@0"
I20260811 09:01:16.710302 12202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:01:16.713625 12218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:16.714833 12202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:01:16.715051 12202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "547fe07a0b3d4c2e86f5045c8637df31"
format_stamp: "Formatted at 2026-08-11 09:01:16 on dist-test-slave-cbsf"
server_key: "af4d971a5ff8be57b8894a0b9c644888"
server_key_iv: "9174d587a762b94fa1a66a0f172e975f"
server_key_version: "encryptionkey@0"
I20260811 09:01:16.715250 12202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:16.729264 12202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:16.733333 12202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:16.733646 12202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:16.754849 12202 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35543
I20260811 09:01:16.754891 12275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35543 every 8 connection(s)
I20260811 09:01:16.756302 12202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:16.760114 12276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:16.762848 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12202
I20260811 09:01:16.763082 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:16.763749 9377 external_mini_cluster.cc:1428] Setting key 8567bd3075d2947d92a36021b64e62a2
I20260811 09:01:16.772122 12276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31: Bootstrap starting.
I20260811 09:01:16.775861 12276 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:16.777209 12276 log.cc:824] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:01:16 dist-test-slave-cbsf krb5kdc[12187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:16.780874 12276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31: No bootstrap required, opened a new log
I20260811 09:01:16.789384 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:16.765698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.9.40.126:35543)
0811 09:01:16.766450 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:16.766462 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:16.766504 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:01:16.768196 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:16.768217 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:16.768583 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:16.768943 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:16.770694 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:16.771933 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:16.773760 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:16.774530 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:16.774663 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:16.779830 (+ 5167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:16.779841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:16.779861 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:16.779919 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:16.784525 (+ 4606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:16.785323 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:16.785332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:16.785342 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:16.785596 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:16.785955 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:16.785960 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:16.785963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:16.786705 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:16.787064 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:16.787381 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 09:01:16.787876 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":373,"threads_started":1}
I20260811 09:01:16.792235 12276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } }
I20260811 09:01:16.792783 12276 raft_consensus.cc:374] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:16.792886 12276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547fe07a0b3d4c2e86f5045c8637df31, State: Initialized, Role: FOLLOWER
I20260811 09:01:16.793403 12276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } }
I20260811 09:01:16.793629 12276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:16.793737 12276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:16.793879 12276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:16.796240 12276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } }
I20260811 09:01:16.796875 12276 leader_election.cc:302] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 547fe07a0b3d4c2e86f5045c8637df31; no voters:
I20260811 09:01:16.797524 12276 leader_election.cc:288] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:16.797665 12283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:16.798409 12283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [term 1 LEADER]: Becoming Leader. State: Replica: 547fe07a0b3d4c2e86f5045c8637df31, State: Running, Role: LEADER
I20260811 09:01:16.798987 12283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } }
I20260811 09:01:16.799813 12276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:16.801833 12285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 547fe07a0b3d4c2e86f5045c8637df31. Latest consensus state: current_term: 1 leader_uuid: "547fe07a0b3d4c2e86f5045c8637df31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } } }
I20260811 09:01:16.802320 12285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:16.802795 12284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "547fe07a0b3d4c2e86f5045c8637df31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547fe07a0b3d4c2e86f5045c8637df31" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35543 } } }
I20260811 09:01:16.803119 12284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:16.806061 12293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:16.811359 12293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:16.821535 12293 catalog_manager.cc:1347] Generated new cluster ID: 1ca454ad4c5f4ee7921424717b32e79b
I20260811 09:01:16.821699 12293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:16.843474 12293 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:16.844869 12293 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:16.850850 12293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 547fe07a0b3d4c2e86f5045c8637df31: Generated new TSK 0
I20260811 09:01:16.851701 12293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:16.924497 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:17.277712 12307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:17.278028 12307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:17.278129 12307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:17.285995 12307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:17.286101 12307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:17.286237 12307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:17.296082 12307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:17.297570 12307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:17.298851 12307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:17.312649 12312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:17.312575 12314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:17.312728 12316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:17.313740 12307 server_base.cc:1034] running on GCE node
I20260811 09:01:17.314419 12307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:17.315424 12307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:17.316632 12307 hybrid_clock.cc:648] HybridClock initialized: now 1786438877316563 us; error 57 us; skew 500 ppm
I20260811 09:01:17.317118 12307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:17 dist-test-slave-cbsf krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438877, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:17.320644 12307 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:17.322178 12307 webserver.cc:460] Webserver started at http://127.9.40.65:34681/ using document root <none> and password file <none>
I20260811 09:01:17.322898 12307 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:17.323004 12307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:17.323378 12307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:17.326498 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "28635e6d667d418199a9b449ae8bae08"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "bf8e11b956ccb90f8d375573c3337287"
server_key_iv: "657e241965389fbc5d175cd4d9d38e16"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.327332 12307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "28635e6d667d418199a9b449ae8bae08"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "bf8e11b956ccb90f8d375573c3337287"
server_key_iv: "657e241965389fbc5d175cd4d9d38e16"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.332638 12307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:01:17.336159 12325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.337585 12307 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:17.337761 12307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "28635e6d667d418199a9b449ae8bae08"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "bf8e11b956ccb90f8d375573c3337287"
server_key_iv: "657e241965389fbc5d175cd4d9d38e16"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.337956 12307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:17.384168 12307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:17.388262 12307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:17.388604 12307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:17.389629 12307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:17.391191 12307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:17.391301 12307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.391454 12307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:17.391526 12307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.427757 12307 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42951
I20260811 09:01:17.427806 12453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42951 every 8 connection(s)
I20260811 09:01:17.429167 12307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:17.431880 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12307
I20260811 09:01:17.432125 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:17.432895 9377 external_mini_cluster.cc:1428] Setting key 95a43b937ce69325a71d7f59e91958ad
Aug 11 09:01:17 dist-test-slave-cbsf krb5kdc[12187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438877, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:17.450794 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:17.431751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:38947 (local address 127.9.40.126:35543)
0811 09:01:17.432292 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:17.432301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:17.433329 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 09:01:17.435496 (+ 2167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:17.435504 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:17.435600 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:17.435727 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:17.437262 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.438456 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.440002 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.440733 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.440887 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:17.444973 (+ 4086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:17.444978 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:17.444981 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:17.445014 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:17.447625 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:17.448352 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:17.448359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:17.448367 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:17.448483 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:17.448996 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:17.449001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:17.449006 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:17.449283 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:17.449446 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:17.450167 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 09:01:17.450404 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":263,"threads_started":1}
I20260811 09:01:17.451804 12457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:17.432321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35543 (local address 127.9.40.65:38947)
0811 09:01:17.433150 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:17.433197 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:17.435227 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:17.435913 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:17.435920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:17.436431 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:17.437075 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:17.437093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.438615 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.438618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:17.439844 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:17.439857 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.440972 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.440980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:17.441209 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:17.441930 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:17.441953 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:17.444742 (+ 2789us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:17.447776 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:17.447785 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:17.447802 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:17.448223 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:17.448602 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:17.448608 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:17.448612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:17.448875 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:17.449433 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:17.449441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:17.449787 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 09:01:17.450044 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 09:01:17.450606 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":366,"threads_started":1}
I20260811 09:01:17.453608 12454 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35543
I20260811 09:01:17.453918 12454 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:01:17.454639 12454 heartbeater.cc:511] Master 127.9.40.126:35543 requested a full tablet report, sending...
I20260811 09:01:17.456959 12239 ts_manager.cc:194] Registered new tserver with Master: 28635e6d667d418199a9b449ae8bae08 (127.9.40.65:42951)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
I20260811 09:01:17.458537 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:38947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:17.482347 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:17.818442 12460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:17.818814 12460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:17.818934 12460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:17.826426 12460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:17.826534 12460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:17.826650 12460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:17.836818 12460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:17.838058 12460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:17.839336 12460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:17.853456 12466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:17.853425 12467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:17.853523 12469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:17.854266 12460 server_base.cc:1034] running on GCE node
I20260811 09:01:17.854887 12460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:17.855755 12460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:17.856972 12460 hybrid_clock.cc:648] HybridClock initialized: now 1786438877856925 us; error 42 us; skew 500 ppm
I20260811 09:01:17.857378 12460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:17 dist-test-slave-cbsf krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438877, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:17.860781 12460 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:17.862334 12460 webserver.cc:460] Webserver started at http://127.9.40.66:34241/ using document root <none> and password file <none>
I20260811 09:01:17.863024 12460 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:17.863175 12460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:17.863492 12460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:17.866493 12460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "0444935258bd44f78de17f97563aee79"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "ce1fcc45d6d9537407cb2d3d8753b010"
server_key_iv: "753b680e4da401ce6b5c79e56cf02487"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.867285 12460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "0444935258bd44f78de17f97563aee79"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "ce1fcc45d6d9537407cb2d3d8753b010"
server_key_iv: "753b680e4da401ce6b5c79e56cf02487"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.872344 12460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:01:17.875736 12478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.877125 12460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:01:17.877307 12460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "0444935258bd44f78de17f97563aee79"
format_stamp: "Formatted at 2026-08-11 09:01:17 on dist-test-slave-cbsf"
server_key: "ce1fcc45d6d9537407cb2d3d8753b010"
server_key_iv: "753b680e4da401ce6b5c79e56cf02487"
server_key_version: "encryptionkey@0"
I20260811 09:01:17.877501 12460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:17.892529 12460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:17.896956 12460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:17.897264 12460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:17.898244 12460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:17.899749 12460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:17.899848 12460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.899996 12460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:17.900070 12460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:17.934532 12460 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35545
I20260811 09:01:17.934590 12607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35545 every 8 connection(s)
I20260811 09:01:17.935854 12460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:17.942960 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12460
I20260811 09:01:17.943146 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:17.943626 9377 external_mini_cluster.cc:1428] Setting key e435e66ffcf3795e2de10717ad799a3a
Aug 11 09:01:17 dist-test-slave-cbsf krb5kdc[12187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438877, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:17.957741 12456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:17.938546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:33283 (local address 127.9.40.126:35543)
0811 09:01:17.938730 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:17.938735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:17.939947 (+ 1212us) server_negotiation.cc:407] Connection header received
0811 09:01:17.941881 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:17.941887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:17.941981 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:17.942110 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:17.943741 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.944630 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.946607 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.947015 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.947114 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:17.951571 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:17.951576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:17.951580 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:17.951612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:17.954225 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:17.954967 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:17.954973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:17.954977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:17.955106 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:17.955562 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:17.955567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:17.955570 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:17.955837 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:17.956043 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:17.956981 (+ 938us) server_negotiation.cc:299] Negotiation successful
0811 09:01:17.957283 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:01:17.958729 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:17.939063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35543 (local address 127.9.40.66:33283)
0811 09:01:17.939787 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:17.939824 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:17.941591 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:17.942337 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:17.942345 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:17.942879 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:17.943547 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:17.943572 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.944861 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.944873 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:17.946450 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:17.946465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:17.947338 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:17.947348 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:17.947742 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:17.948495 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:17.948530 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:17.951335 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:17.954387 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:17.954397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:17.954414 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:17.954846 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:17.955218 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:17.955223 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:17.955227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:17.955425 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:17.956015 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:17.956023 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:17.956381 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 09:01:17.956793 (+ 412us) client_negotiation.cc:240] Negotiation successful
0811 09:01:17.957477 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":282,"threads_started":1}
I20260811 09:01:17.960282 12608 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35543
I20260811 09:01:17.960620 12608 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:17.961385 12608 heartbeater.cc:511] Master 127.9.40.126:35543 requested a full tablet report, sending...
I20260811 09:01:17.962919 12239 ts_manager.cc:194] Registered new tserver with Master: 0444935258bd44f78de17f97563aee79 (127.9.40.66:35545)
I20260811 09:01:17.963732 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:33283
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:17.994370 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:18.331872 12614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:18.332170 12614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:18.332360 12614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:18.340235 12614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:18.340363 12614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:18.340608 12614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:18.350596 12614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:35543
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:18.351967 12614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:18.353466 12614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:18.367086 12622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:18.367144 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:18.367166 12624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:18.368019 12614 server_base.cc:1034] running on GCE node
I20260811 09:01:18.368765 12614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:18.369789 12614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:18.371047 12614 hybrid_clock.cc:648] HybridClock initialized: now 1786438878370983 us; error 43 us; skew 500 ppm
I20260811 09:01:18.371402 12614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:18 dist-test-slave-cbsf krb5kdc[12187](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438878, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:18.374805 12614 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:18.376338 12614 webserver.cc:460] Webserver started at http://127.9.40.67:35533/ using document root <none> and password file <none>
I20260811 09:01:18.377105 12614 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:18.377271 12614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:18.377606 12614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:18.380589 12614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "4d98ce0bfde1415c8573b0dd20e9ae71"
format_stamp: "Formatted at 2026-08-11 09:01:18 on dist-test-slave-cbsf"
server_key: "c0d2a73baa2396f4f3bb9fc7bf6bd6f5"
server_key_iv: "e2e2e8fa97430c09001768604de7aca6"
server_key_version: "encryptionkey@0"
I20260811 09:01:18.381474 12614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "4d98ce0bfde1415c8573b0dd20e9ae71"
format_stamp: "Formatted at 2026-08-11 09:01:18 on dist-test-slave-cbsf"
server_key: "c0d2a73baa2396f4f3bb9fc7bf6bd6f5"
server_key_iv: "e2e2e8fa97430c09001768604de7aca6"
server_key_version: "encryptionkey@0"
I20260811 09:01:18.386585 12614 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:01:18.390125 12632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:18.391453 12614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:18.391642 12614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "4d98ce0bfde1415c8573b0dd20e9ae71"
format_stamp: "Formatted at 2026-08-11 09:01:18 on dist-test-slave-cbsf"
server_key: "c0d2a73baa2396f4f3bb9fc7bf6bd6f5"
server_key_iv: "e2e2e8fa97430c09001768604de7aca6"
server_key_version: "encryptionkey@0"
I20260811 09:01:18.391848 12614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:18.412412 12614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:18.417315 12614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:18.417675 12614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:18.418988 12614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:18.420423 12614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:18.420534 12614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:18.420696 12614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:18.420783 12614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 09:01:18.456686 12614 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35801
I20260811 09:01:18.456804 12755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35801 every 8 connection(s)
I20260811 09:01:18.458043 12614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:18.461357 12454 heartbeater.cc:503] Master 127.9.40.126:35543 was elected leader, sending a full tablet report...
I20260811 09:01:18.466627 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12614
I20260811 09:01:18.466800 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:18.467290 9377 external_mini_cluster.cc:1428] Setting key eaf88d118009bcded991b5ed9541fcdf
Aug 11 09:01:18 dist-test-slave-cbsf krb5kdc[12187](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438878, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:18.478470 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:18.460748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36923 (local address 127.9.40.126:35543)
0811 09:01:18.461265 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:18.461273 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:18.462370 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 09:01:18.464402 (+ 2032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:18.464408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:18.464489 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:18.464622 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:18.466150 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:18.467076 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:18.469096 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:18.469664 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:18.469792 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:18.473257 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:18.473262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:18.473266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:18.473297 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:18.475748 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:18.476408 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:18.476411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:18.476414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:18.476524 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:18.476980 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:18.476983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:18.476985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:18.477262 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:18.477372 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:18.478033 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 09:01:18.478204 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":235,"threads_started":1}
I20260811 09:01:18.479404 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:18.461339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35543 (local address 127.9.40.67:36923)
0811 09:01:18.462221 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:18.462268 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:18.464146 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:18.464903 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:18.464910 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:18.465426 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:18.465979 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:18.465996 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:18.467214 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:18.467220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:18.468948 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:18.468960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:18.469887 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:18.469892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:18.470132 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:18.470836 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:18.470858 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:18.473048 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:18.475871 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:18.475878 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:18.475889 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:18.476298 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:18.476616 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:18.476619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:18.476621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:18.476868 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:18.477369 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:18.477375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:18.477702 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 09:01:18.477921 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 09:01:18.478306 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":343,"threads_started":1}
I20260811 09:01:18.480763 12756 heartbeater.cc:347] Connected to a master server at 127.9.40.126:35543
I20260811 09:01:18.481105 12756 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:18.482025 12756 heartbeater.cc:511] Master 127.9.40.126:35543 requested a full tablet report, sending...
I20260811 09:01:18.483481 12239 ts_manager.cc:194] Registered new tserver with Master: 4d98ce0bfde1415c8573b0dd20e9ae71 (127.9.40.67:35801)
I20260811 09:01:18.484390 12239 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:36923
I20260811 09:01:18.495088 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:01:18.531576 12758 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:18.529069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.9.40.126:35543)
0811 09:01:18.529339 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:18.529347 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:18.529683 (+ 336us) server_negotiation.cc:407] Connection header received
0811 09:01:18.529830 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:18.529833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:18.529903 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:18.530000 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:18.531281 (+ 1281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42128: BlockingRecv error: recv got EOF from 127.0.0.1:42128 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:01:18.534726 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12307
W20260811 09:01:18.537683 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12307: Not authorized: Client connection negotiation failed: client connection to 127.9.40.65:42951: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:01:18.537886 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12307
I20260811 09:01:18.557914 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12460
W20260811 09:01:18.560526 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12460: Not authorized: Client connection negotiation failed: client connection to 127.9.40.66:35545: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:01:18.560668 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12460
I20260811 09:01:18.580137 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12614
W20260811 09:01:18.583395 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12614: Not authorized: Client connection negotiation failed: client connection to 127.9.40.67:35801: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:01:18.583607 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12614
I20260811 09:01:18.604553 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12202
I20260811 09:01:18.605890 12271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:18.709631 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12202
2026-08-11T09:01:18Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3869 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:18 dist-test-slave-cbsf krb5kdc[12777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:18 dist-test-slave-cbsf krb5kdc[12777](info): set up 2 sockets
Aug 11 09:01:18 dist-test-slave-cbsf krb5kdc[12777](info): commencing operation
krb5kdc: starting...
W20260811 09:01:19.874488 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:20.089025 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42285
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:45345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:42285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:20.422160 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:20.422514 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:20.422696 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:20.430894 12790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:20.431001 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:20.431084 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:20.431170 12790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:20.431259 12790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:20.441706 12790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:45345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:42285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:20.443011 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:20.444399 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:20.456673 12795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:20.456789 12799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:20.456784 12796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:20.457849 12790 server_base.cc:1034] running on GCE node
I20260811 09:01:20.458714 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:20.460140 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:20.461350 12790 hybrid_clock.cc:648] HybridClock initialized: now 1786438880461296 us; error 51 us; skew 500 ppm
I20260811 09:01:20.461773 12790 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438880, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:20.470798 12790 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:20.472467 12790 webserver.cc:460] Webserver started at http://127.9.40.126:42663/ using document root <none> and password file <none>
I20260811 09:01:20.473335 12790 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:20.473498 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:20.473834 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:20.476925 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "e5a9b4581e354f3887d1a40178d63c07"
format_stamp: "Formatted at 2026-08-11 09:01:20 on dist-test-slave-cbsf"
server_key: "faf635457eea61ea57e36b841ff050a5"
server_key_iv: "de086f826b4b54859fbc320c483aca2e"
server_key_version: "encryptionkey@0"
I20260811 09:01:20.477756 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "e5a9b4581e354f3887d1a40178d63c07"
format_stamp: "Formatted at 2026-08-11 09:01:20 on dist-test-slave-cbsf"
server_key: "faf635457eea61ea57e36b841ff050a5"
server_key_iv: "de086f826b4b54859fbc320c483aca2e"
server_key_version: "encryptionkey@0"
I20260811 09:01:20.483011 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:01:20.486451 12807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:20.487764 12790 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:01:20.487949 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "e5a9b4581e354f3887d1a40178d63c07"
format_stamp: "Formatted at 2026-08-11 09:01:20 on dist-test-slave-cbsf"
server_key: "faf635457eea61ea57e36b841ff050a5"
server_key_iv: "de086f826b4b54859fbc320c483aca2e"
server_key_version: "encryptionkey@0"
I20260811 09:01:20.488152 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:20.528990 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:20.533357 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:20.533699 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:20.553450 12790 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:42285
I20260811 09:01:20.553494 12862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:42285 every 8 connection(s)
I20260811 09:01:20.554811 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:20.559059 12863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:20.560253 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12790
I20260811 09:01:20.560520 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:20.561430 9377 external_mini_cluster.cc:1428] Setting key d0dc1f6f54c04bc07dc941ae35da7a8f
I20260811 09:01:20.572032 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07: Bootstrap starting.
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:20.575239 12863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:20.576296 12863 log.cc:824] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:20.579360 12863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07: No bootstrap required, opened a new log
I20260811 09:01:20.583655 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:20.563323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39110 (local address 127.9.40.126:42285)
0811 09:01:20.564087 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:20.564099 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:20.564139 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:01:20.565909 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:20.565931 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:20.566301 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:20.566698 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:20.568271 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:20.569699 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:20.570954 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:20.571617 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:20.571813 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:20.576073 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:20.576080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:20.576094 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:20.576139 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:20.579610 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:20.580283 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:20.580291 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:20.580299 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:20.580466 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:20.580924 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:20.580930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:20.580937 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:20.581489 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:20.581835 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:20.582285 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 09:01:20.582699 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":331,"threads_started":1}
I20260811 09:01:20.587533 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:01:20.587955 12863 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } }
I20260811 09:01:20.588340 12863 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:20.588449 12863 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5a9b4581e354f3887d1a40178d63c07, State: Initialized, Role: FOLLOWER
I20260811 09:01:20.589028 12863 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } }
I20260811 09:01:20.589274 12863 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:20.589416 12863 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:20.589596 12863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:20.591740 12863 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } }
I20260811 09:01:20.592260 12863 leader_election.cc:302] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5a9b4581e354f3887d1a40178d63c07; no voters:
I20260811 09:01:20.592973 12863 leader_election.cc:288] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:20.593061 12869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:20.595466 12863 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:20.596554 12869 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [term 1 LEADER]: Becoming Leader. State: Replica: e5a9b4581e354f3887d1a40178d63c07, State: Running, Role: LEADER
I20260811 09:01:20.597244 12869 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } }
I20260811 09:01:20.601006 12872 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5a9b4581e354f3887d1a40178d63c07. Latest consensus state: current_term: 1 leader_uuid: "e5a9b4581e354f3887d1a40178d63c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } } }
I20260811 09:01:20.601446 12872 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:20.602065 12870 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5a9b4581e354f3887d1a40178d63c07" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5a9b4581e354f3887d1a40178d63c07" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42285 } } }
I20260811 09:01:20.602406 12870 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:20.612306 12884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:20.619103 12884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:20.628062 12891 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:01:20.633898 12884 catalog_manager.cc:1347] Generated new cluster ID: c95eeca6fd3642abbce9ea17579bc9b8
I20260811 09:01:20.634022 12884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:20.659967 12866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:20.636768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.9.40.126:42285)
0811 09:01:20.636931 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:20.636937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:20.650884 (+ 13947us) server_negotiation.cc:407] Connection header received
0811 09:01:20.651085 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:20.651092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:20.651176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:20.651253 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:20.652741 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:20.653801 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:20.655798 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:20.656500 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:20.656618 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:20.658032 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:20.658038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:20.658043 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:20.659121 (+ 1078us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:20.659197 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:20.659229 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:20.659336 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:20.659514 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:01:20.659612 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:01:20.667771 12884 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:20.669298 12884 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:20.680541 12884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5a9b4581e354f3887d1a40178d63c07: Generated new TSK 0
I20260811 09:01:20.681421 12884 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:01:20.689961 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12790
I20260811 09:01:20.691118 12858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:20.784183 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12790
2026-08-11T09:01:20Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2069 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12903](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12903](info): set up 2 sockets
Aug 11 09:01:20 dist-test-slave-cbsf krb5kdc[12903](info): commencing operation
krb5kdc: starting...
W20260811 09:01:21.946449 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:22 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:22.160017 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46383
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:22.499236 12917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:22.499550 12917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:22.499750 12917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:22.507186 12917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:22.507309 12917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:22.507463 12917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:22.507550 12917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:22.507642 12917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:22.517788 12917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:22.519167 12917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:22.520524 12917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:22.533218 12923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:22.533136 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:22.533309 12924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:22.534217 12917 server_base.cc:1034] running on GCE node
I20260811 09:01:22.535101 12917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:22.536492 12917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:22.537746 12917 hybrid_clock.cc:648] HybridClock initialized: now 1786438882537712 us; error 41 us; skew 500 ppm
I20260811 09:01:22.538136 12917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:22 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438882, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:22.541532 12917 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:22.543124 12917 webserver.cc:460] Webserver started at http://127.9.40.126:40391/ using document root <none> and password file <none>
I20260811 09:01:22.543824 12917 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:22.543995 12917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:22.544329 12917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:22.547514 12917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "319d5d73416643eca3cdd8c5b148ae74"
format_stamp: "Formatted at 2026-08-11 09:01:22 on dist-test-slave-cbsf"
server_key: "2f22d250cee8a02ed84c8e0fe23c8b73"
server_key_iv: "3f3f6cd9c47d4c13b91e48c4f40bcda0"
server_key_version: "encryptionkey@0"
I20260811 09:01:22.548322 12917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "319d5d73416643eca3cdd8c5b148ae74"
format_stamp: "Formatted at 2026-08-11 09:01:22 on dist-test-slave-cbsf"
server_key: "2f22d250cee8a02ed84c8e0fe23c8b73"
server_key_iv: "3f3f6cd9c47d4c13b91e48c4f40bcda0"
server_key_version: "encryptionkey@0"
I20260811 09:01:22.553848 12917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:01:22.557384 12934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:22.558640 12917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:22.558835 12917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "319d5d73416643eca3cdd8c5b148ae74"
format_stamp: "Formatted at 2026-08-11 09:01:22 on dist-test-slave-cbsf"
server_key: "2f22d250cee8a02ed84c8e0fe23c8b73"
server_key_iv: "3f3f6cd9c47d4c13b91e48c4f40bcda0"
server_key_version: "encryptionkey@0"
I20260811 09:01:22.559032 12917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:22.573316 12917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:22.577121 12917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:22.577416 12917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:22.596426 12917 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46383
I20260811 09:01:22.596508 12992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46383 every 8 connection(s)
I20260811 09:01:22.597798 12917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:22.600847 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 12917
I20260811 09:01:22.601109 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:22.601890 12993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:22.601984 9377 external_mini_cluster.cc:1428] Setting key 0508f87ae4c28a04f266a425c816a159
I20260811 09:01:22.614331 12993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74: Bootstrap starting.
Aug 11 09:01:22 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:22.617956 12993 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:22.619032 12993 log.cc:824] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:22.622596 12993 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74: No bootstrap required, opened a new log
I20260811 09:01:22.624856 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:22.604108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51210 (local address 127.9.40.126:46383)
0811 09:01:22.604958 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:22.604973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:22.605015 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:01:22.606747 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:22.606768 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:22.607123 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:22.607506 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:22.608984 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:22.610236 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:22.611611 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:22.612303 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:22.612553 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:22.617068 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:22.617075 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:22.617088 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:22.617123 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:22.620828 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:22.621637 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:22.621643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:22.621649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:22.621804 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:22.622214 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:22.622217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:22.622219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:22.622759 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:22.622996 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:22.623433 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 09:01:22.623776 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":356,"threads_started":1}
I20260811 09:01:22.630483 12993 raft_consensus.cc:348] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } }
I20260811 09:01:22.630842 12993 raft_consensus.cc:374] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:22.630970 12993 raft_consensus.cc:729] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319d5d73416643eca3cdd8c5b148ae74, State: Initialized, Role: FOLLOWER
I20260811 09:01:22.631487 12993 consensus_queue.cc:260] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } }
I20260811 09:01:22.631724 12993 raft_consensus.cc:388] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:22.631872 12993 raft_consensus.cc:482] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:22.632055 12993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:22.634280 12993 raft_consensus.cc:504] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } }
I20260811 09:01:22.634768 12993 leader_election.cc:302] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 319d5d73416643eca3cdd8c5b148ae74; no voters:
I20260811 09:01:22.635321 12993 leader_election.cc:288] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:22.635450 12999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:22.636147 12999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [term 1 LEADER]: Becoming Leader. State: Replica: 319d5d73416643eca3cdd8c5b148ae74, State: Running, Role: LEADER
I20260811 09:01:22.636680 12999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } }
I20260811 09:01:22.637435 12993 sys_catalog.cc:564] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:22.639379 13000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "319d5d73416643eca3cdd8c5b148ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } } }
I20260811 09:01:22.639835 13000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:22.639722 13001 sys_catalog.cc:455] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 319d5d73416643eca3cdd8c5b148ae74. Latest consensus state: current_term: 1 leader_uuid: "319d5d73416643eca3cdd8c5b148ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319d5d73416643eca3cdd8c5b148ae74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46383 } } }
I20260811 09:01:22.640007 13001 sys_catalog.cc:458] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:22.642127 13004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:22.647363 13004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:22.656162 13004 catalog_manager.cc:1347] Generated new cluster ID: ab7e578e965545bab7027216235fe6bc
I20260811 09:01:22.656289 13004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:22.667582 13004 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:22.668999 13004 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:22.675468 13004 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 319d5d73416643eca3cdd8c5b148ae74: Generated new TSK 0
I20260811 09:01:22.676242 13004 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:22.732237 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:23.095882 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:23.096186 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:23.096295 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:23.103653 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:23.103770 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:23.103890 13022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:23.114140 13022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:23.115453 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:23.116758 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:23.130364 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:23.130411 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:23.130450 13031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:23.134308 13022 server_base.cc:1034] running on GCE node
I20260811 09:01:23.134936 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:23.135902 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:23.137094 13022 hybrid_clock.cc:648] HybridClock initialized: now 1786438883137029 us; error 57 us; skew 500 ppm
I20260811 09:01:23.137415 13022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:23 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438883, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:23.140935 13022 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:23.142513 13022 webserver.cc:460] Webserver started at http://127.9.40.65:42365/ using document root <none> and password file <none>
I20260811 09:01:23.143297 13022 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:23.143460 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:23.143826 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:23.146961 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "11ca516b3ed0454fb888df3a66b8d770"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "210b85049ca13e22ae5b12ad0921b930"
server_key_iv: "ee35885d74fb7d8dc689bfc46f77a113"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.147785 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "11ca516b3ed0454fb888df3a66b8d770"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "210b85049ca13e22ae5b12ad0921b930"
server_key_iv: "ee35885d74fb7d8dc689bfc46f77a113"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.153100 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:01:23.156579 13038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.158161 13022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:23.158345 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "11ca516b3ed0454fb888df3a66b8d770"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "210b85049ca13e22ae5b12ad0921b930"
server_key_iv: "ee35885d74fb7d8dc689bfc46f77a113"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.158569 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:23.194404 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:23.198591 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:23.198916 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:23.199937 13022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:23.201386 13022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:23.201493 13022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.201659 13022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:23.201735 13022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.237190 13022 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37311
I20260811 09:01:23.237253 13170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37311 every 8 connection(s)
I20260811 09:01:23.238607 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:23.240314 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13022
I20260811 09:01:23.240551 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:23.241480 9377 external_mini_cluster.cc:1428] Setting key 0b21af2eb68b140884713887230b931a
Aug 11 09:01:23 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438883, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:23.261632 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:23.242189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39523 (local address 127.9.40.126:46383)
0811 09:01:23.242681 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:23.242689 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:23.243813 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 09:01:23.245816 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:23.245826 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:23.245918 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:23.246045 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:23.247578 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.248785 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.250354 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.251048 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.251192 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:23.255265 (+ 4073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:23.255270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:23.255273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:23.255304 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:23.258025 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:23.258801 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:23.258806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:23.258809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:23.258934 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:23.259644 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:23.259651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:23.259656 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:23.259966 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:23.260136 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:23.260982 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 09:01:23.261220 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":239,"threads_started":1}
I20260811 09:01:23.262888 13175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:23.242784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46383 (local address 127.9.40.65:39523)
0811 09:01:23.243633 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:23.243696 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:23.245544 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:23.246245 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:23.246253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:23.246815 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:23.247385 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:23.247402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.248938 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.248942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:23.250199 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:23.250212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.251255 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.251260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:23.251466 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:23.252238 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:23.252278 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:23.255011 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:23.258220 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:23.258229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:23.258245 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:23.258668 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:23.259049 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:23.259054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:23.259058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:23.259500 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:23.260108 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:23.260116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:23.260508 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 09:01:23.260865 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 09:01:23.261417 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":319,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:01:23.264680 13171 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46383
I20260811 09:01:23.265018 13171 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:23.265789 13171 heartbeater.cc:511] Master 127.9.40.126:46383 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
I20260811 09:01:23.267989 12955 ts_manager.cc:194] Registered new tserver with Master: 11ca516b3ed0454fb888df3a66b8d770 (127.9.40.65:37311)
I20260811 09:01:23.269539 12955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:39523
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:23.304242 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:23.639820 13178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:23.640098 13178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:23.640197 13178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:23.647343 13178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:23.647451 13178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:23.647661 13178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:23.658188 13178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:23.659487 13178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:23.660833 13178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:23.674377 13183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:23.674407 13184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:23.674947 13186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:23.675380 13178 server_base.cc:1034] running on GCE node
I20260811 09:01:23.676062 13178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:23.676937 13178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:23.678135 13178 hybrid_clock.cc:648] HybridClock initialized: now 1786438883678085 us; error 43 us; skew 500 ppm
I20260811 09:01:23.678454 13178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:23 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438883, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:23.681553 13178 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:23.682916 13178 webserver.cc:460] Webserver started at http://127.9.40.66:44533/ using document root <none> and password file <none>
I20260811 09:01:23.683606 13178 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:23.683754 13178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:23.684083 13178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:23.687017 13178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "b15f1c5d270a4421b0ff2743b8eaaad8"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "bf624e9caa15af7c5fef9e9277cd60a2"
server_key_iv: "8f206fb3a8de645aafe3c9e72e9ba4c8"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.687805 13178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "b15f1c5d270a4421b0ff2743b8eaaad8"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "bf624e9caa15af7c5fef9e9277cd60a2"
server_key_iv: "8f206fb3a8de645aafe3c9e72e9ba4c8"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.692879 13178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 09:01:23.696298 13197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.697762 13178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:23.697958 13178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "b15f1c5d270a4421b0ff2743b8eaaad8"
format_stamp: "Formatted at 2026-08-11 09:01:23 on dist-test-slave-cbsf"
server_key: "bf624e9caa15af7c5fef9e9277cd60a2"
server_key_iv: "8f206fb3a8de645aafe3c9e72e9ba4c8"
server_key_version: "encryptionkey@0"
I20260811 09:01:23.698161 13178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:23.719221 13178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:23.723007 13178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:23.723344 13178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:23.724282 13178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:23.725859 13178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:23.725996 13178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.726146 13178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:23.726195 13178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:23.760748 13178 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35195
I20260811 09:01:23.760812 13324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35195 every 8 connection(s)
I20260811 09:01:23.762418 13178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:23.765566 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13178
I20260811 09:01:23.765775 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:23.766429 9377 external_mini_cluster.cc:1428] Setting key 954864b6803f855675c5b4b85de74a88
Aug 11 09:01:23 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438883, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:23.784935 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:23.765212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:33807 (local address 127.9.40.126:46383)
0811 09:01:23.765758 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:23.765763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:23.766888 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 09:01:23.769048 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:23.769053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:23.769155 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:23.769269 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:23.770695 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.772006 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.773669 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.774103 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.774243 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:23.777790 (+ 3547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:23.777799 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:23.777807 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:23.777850 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:23.781398 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:23.782216 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:23.782224 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:23.782231 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:23.782400 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:23.782827 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:23.782834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:23.782840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:23.783242 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:23.783398 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:23.784049 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 09:01:23.784339 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":309,"threads_started":1}
I20260811 09:01:23.785571 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:23.765872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46383 (local address 127.9.40.66:33807)
0811 09:01:23.766739 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:23.766777 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:23.768788 (+ 2011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:23.769501 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:23.769509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:23.769987 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:23.770526 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:23.770542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.772174 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.772178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:23.773514 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:23.773526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:23.774331 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:23.774340 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:23.774596 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:23.775277 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:23.775299 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:23.777565 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:23.781564 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:23.781571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:23.781583 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:23.782053 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:23.782509 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:23.782512 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:23.782514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:23.782697 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:23.783363 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:23.783368 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:23.783674 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 09:01:23.783893 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 09:01:23.784264 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":324,"threads_started":1}
I20260811 09:01:23.787179 13326 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46383
I20260811 09:01:23.787544 13326 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:23.788573 13326 heartbeater.cc:511] Master 127.9.40.126:46383 requested a full tablet report, sending...
I20260811 09:01:23.790254 12955 ts_manager.cc:194] Registered new tserver with Master: b15f1c5d270a4421b0ff2743b8eaaad8 (127.9.40.66:35195)
I20260811 09:01:23.791110 12955 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:33807
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:23.825752 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:24.155925 13332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:24.156252 13332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:24.156433 13332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:24.163969 13332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:24.164108 13332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:24.164287 13332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:24.173839 13332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46383
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:24.175171 13332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:24.176496 13332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:24.189548 13340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:24.189560 13339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:24.189795 13342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:24.190503 13332 server_base.cc:1034] running on GCE node
I20260811 09:01:24.191044 13332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:24.191885 13332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:24.193073 13332 hybrid_clock.cc:648] HybridClock initialized: now 1786438884193039 us; error 25 us; skew 500 ppm
I20260811 09:01:24.193351 13332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:24.196483 13332 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:24.197832 13332 webserver.cc:460] Webserver started at http://127.9.40.67:35211/ using document root <none> and password file <none>
I20260811 09:01:24.198465 13332 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:24.198559 13332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:24.198817 13332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:24.201782 13332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "f19601ebd2ba4e72bee273b07f8af910"
format_stamp: "Formatted at 2026-08-11 09:01:24 on dist-test-slave-cbsf"
server_key: "9e23fc3d90847f0284ddfb24e3168c1a"
server_key_iv: "1cf5410f500edd0c698637d886488b5c"
server_key_version: "encryptionkey@0"
I20260811 09:01:24.202649 13332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "f19601ebd2ba4e72bee273b07f8af910"
format_stamp: "Formatted at 2026-08-11 09:01:24 on dist-test-slave-cbsf"
server_key: "9e23fc3d90847f0284ddfb24e3168c1a"
server_key_iv: "1cf5410f500edd0c698637d886488b5c"
server_key_version: "encryptionkey@0"
I20260811 09:01:24.207621 13332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:01:24.211108 13349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:24.212316 13332 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:01:24.212502 13332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "f19601ebd2ba4e72bee273b07f8af910"
format_stamp: "Formatted at 2026-08-11 09:01:24 on dist-test-slave-cbsf"
server_key: "9e23fc3d90847f0284ddfb24e3168c1a"
server_key_iv: "1cf5410f500edd0c698637d886488b5c"
server_key_version: "encryptionkey@0"
I20260811 09:01:24.212697 13332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:24.227505 13332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:24.231176 13332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:24.231513 13332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:24.232407 13332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:24.233769 13332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:24.233842 13332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:24.234010 13332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:24.234064 13332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:24.269387 13332 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:45553
I20260811 09:01:24.269403 13476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:45553 every 8 connection(s)
I20260811 09:01:24.270684 13332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:24.272392 13171 heartbeater.cc:503] Master 127.9.40.126:46383 was elected leader, sending a full tablet report...
I20260811 09:01:24.277083 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13332
I20260811 09:01:24.277345 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:24.277951 9377 external_mini_cluster.cc:1428] Setting key b409d617baae5528aef7d10ec93ca630
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:24.290827 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.273352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45847 (local address 127.9.40.126:46383)
0811 09:01:24.273570 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:24.273577 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:24.274739 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 09:01:24.276481 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:24.276485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:24.276556 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:24.276645 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:24.278813 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.279786 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.281437 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.281853 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.281997 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.285462 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:24.285466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:24.285470 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:24.285501 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:24.288111 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.288906 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.288910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.288912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.289026 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.289397 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.289400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.289402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.289666 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:24.289774 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:24.290415 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 09:01:24.290605 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:01:24.291970 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.273743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46383 (local address 127.9.40.67:45847)
0811 09:01:24.274570 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:24.274614 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:24.276214 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:24.276983 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:24.276994 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:24.277834 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:24.278635 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:24.278655 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.279960 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.279965 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:24.281249 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:24.281261 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.282086 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.282093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:24.282334 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.283043 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:24.283065 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:24.285270 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:24.288246 (+ 2976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:24.288253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:24.288265 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:24.288798 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:24.289117 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:24.289120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:24.289122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:24.289297 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:24.289772 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:24.289777 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:24.290067 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 09:01:24.290316 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 09:01:24.290717 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":346,"threads_started":1}
I20260811 09:01:24.293576 13477 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46383
I20260811 09:01:24.293875 13477 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:24.294610 13477 heartbeater.cc:511] Master 127.9.40.126:46383 requested a full tablet report, sending...
I20260811 09:01:24.295938 12953 ts_manager.cc:194] Registered new tserver with Master: f19601ebd2ba4e72bee273b07f8af910 (127.9.40.67:45553)
I20260811 09:01:24.296828 12953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:45847
I20260811 09:01:24.305400 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:24.361881 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.344201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.9.40.126:46383)
0811 09:01:24.344376 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:24.344381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:24.344971 (+ 590us) server_negotiation.cc:407] Connection header received
0811 09:01:24.345195 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:24.345200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:24.345286 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:24.345405 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:24.346841 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.347612 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.348931 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.349378 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.349516 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.356865 (+ 7349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:24.356870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:24.356873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:24.356909 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:24.359362 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.359981 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.359984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.359987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.360122 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.360491 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.360494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.360496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.360921 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:24.361115 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:24.361419 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 09:01:24.361623 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:01:24.362592 12953 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51226
I20260811 09:01:24.364713 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13022
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:24.380622 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.365835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.9.40.65:37311)
0811 09:01:24.366539 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:24.366546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:24.366571 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:01:24.366624 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:24.366630 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:24.366843 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:24.366975 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:24.368099 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.369270 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.370421 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.371028 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.371129 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.375012 (+ 3883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:24.375024 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:24.375039 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:24.375072 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:24.377952 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.378607 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.378614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.378617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.378742 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.379156 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.379160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.379162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.379537 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:24.379782 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:24.380001 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:01:24.380193 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":381,"threads_started":1}
W20260811 09:01:24.381945 13164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60388
W20260811 09:01:24.383127 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:24.383378 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13022
I20260811 09:01:24.403954 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13178
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:24.419698 13490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.404851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.9.40.66:35195)
0811 09:01:24.405386 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:24.405393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:24.405420 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:01:24.405486 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:24.405492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:24.405702 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:24.405836 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:24.407030 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.408015 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.409318 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.409948 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.410048 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.414092 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:24.414103 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:24.414117 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:24.414152 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:24.417277 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.417936 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.417943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.417945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.418083 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.418397 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.418400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.418402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.418747 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:24.418972 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:24.419202 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 09:01:24.419402 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":289,"threads_started":1}
W20260811 09:01:24.421066 13319 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44126
W20260811 09:01:24.422061 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13178: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:24.422255 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13178
I20260811 09:01:24.443591 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13332
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[12903](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438884, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:24.460314 13491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:24.444647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.9.40.67:45553)
0811 09:01:24.445336 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:24.445344 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:24.445373 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:24.445430 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:24.445437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:24.445655 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:24.445835 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:24.447235 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.448356 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.449656 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:24.450229 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:24.450387 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:24.454728 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:24.454739 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:24.454755 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:24.454788 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:24.457807 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.458416 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.458422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.458425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.458542 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:24.458861 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:24.458864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:24.458866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:24.459191 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:24.459418 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:24.459639 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 09:01:24.459834 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":351,"threads_started":1}
W20260811 09:01:24.461817 13472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37982
W20260811 09:01:24.463003 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13332: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:01:24.463137 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13332
I20260811 09:01:24.482517 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 12917
I20260811 09:01:24.483634 12987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:24.591791 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 12917
2026-08-11T09:01:24Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3811 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[13495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[13495](info): set up 2 sockets
Aug 11 09:01:24 dist-test-slave-cbsf krb5kdc[13495](info): commencing operation
krb5kdc: starting...
W20260811 09:01:25.777326 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.005s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:25 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:25.994706 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36817
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:26.323848 13509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:26.324196 13509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:26.324355 13509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:26.331465 13509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:26.331604 13509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:26.331712 13509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:26.331816 13509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:26.331892 13509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:26.341681 13509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:26.343055 13509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:26.344380 13509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:26.356809 13514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:26.356765 13517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:26.356837 13515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:26.357834 13509 server_base.cc:1034] running on GCE node
I20260811 09:01:26.358639 13509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:26.360149 13509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:26.361403 13509 hybrid_clock.cc:648] HybridClock initialized: now 1786438886361381 us; error 71 us; skew 500 ppm
I20260811 09:01:26.361770 13509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:26 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438886, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:26.368037 13509 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:26.369511 13509 webserver.cc:460] Webserver started at http://127.9.40.126:46389/ using document root <none> and password file <none>
I20260811 09:01:26.370172 13509 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:26.370301 13509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:26.370628 13509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:26.373641 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "4e3a7c79c18c4577bdb3f487d94a3db4"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "45bce3315251481cc94a39fed86c332c"
server_key_iv: "70b87035cd727ab06b834bb29b4c9c78"
server_key_version: "encryptionkey@0"
I20260811 09:01:26.374428 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "4e3a7c79c18c4577bdb3f487d94a3db4"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "45bce3315251481cc94a39fed86c332c"
server_key_iv: "70b87035cd727ab06b834bb29b4c9c78"
server_key_version: "encryptionkey@0"
I20260811 09:01:26.379621 13509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:01:26.382897 13525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:26.384370 13509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:26.384564 13509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "4e3a7c79c18c4577bdb3f487d94a3db4"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "45bce3315251481cc94a39fed86c332c"
server_key_iv: "70b87035cd727ab06b834bb29b4c9c78"
server_key_version: "encryptionkey@0"
I20260811 09:01:26.384806 13509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:26.401360 13509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:26.405411 13509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:26.405730 13509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:26.424892 13509 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36817
I20260811 09:01:26.424914 13586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36817 every 8 connection(s)
I20260811 09:01:26.426340 13509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:26.430038 13587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:26.434705 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13509
I20260811 09:01:26.434938 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:26.435827 9377 external_mini_cluster.cc:1428] Setting key 6f96c91b787b6236e36013d4f2461906
I20260811 09:01:26.441831 13587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4: Bootstrap starting.
I20260811 09:01:26.445415 13587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:26.446664 13587 log.cc:824] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:26.450536 13587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4: No bootstrap required, opened a new log
Aug 11 09:01:26 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:26.459371 13587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } }
I20260811 09:01:26.459800 13587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:26.459932 13587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3a7c79c18c4577bdb3f487d94a3db4, State: Initialized, Role: FOLLOWER
I20260811 09:01:26.460523 13587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } }
I20260811 09:01:26.460793 13587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:26.460940 13587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:26.461097 13587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:26.463344 13587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } }
I20260811 09:01:26.463925 13587 leader_election.cc:302] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e3a7c79c18c4577bdb3f487d94a3db4; no voters:
I20260811 09:01:26.464603 13587 leader_election.cc:288] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:26.464733 13592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:26.465166 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:26.437901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.9.40.126:36817)
0811 09:01:26.438762 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:26.438773 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:26.438814 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:01:26.440664 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:26.440742 (+ 78us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:26.441123 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:26.441426 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:26.443254 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:26.444620 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:26.446520 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:26.447775 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:26.448005 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:26.456765 (+ 8760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:26.456782 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:26.456814 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:26.456870 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:26.460633 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:26.461457 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:26.461465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:26.461474 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:26.461653 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:26.462043 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:26.462048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:26.462051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:26.462611 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:26.462943 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:26.463374 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 09:01:26.463810 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":384,"threads_started":1}
I20260811 09:01:26.466080 13592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [term 1 LEADER]: Becoming Leader. State: Replica: 4e3a7c79c18c4577bdb3f487d94a3db4, State: Running, Role: LEADER
I20260811 09:01:26.466773 13592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } }
I20260811 09:01:26.467093 13587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:26.470247 13593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } } }
I20260811 09:01:26.470693 13593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:26.476554 13594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e3a7c79c18c4577bdb3f487d94a3db4. Latest consensus state: current_term: 1 leader_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e3a7c79c18c4577bdb3f487d94a3db4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36817 } } }
I20260811 09:01:26.476925 13594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:26.480350 13599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:26.484629 13599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:26.493121 13599 catalog_manager.cc:1347] Generated new cluster ID: 0d99b7abf16a410080a0403df3c80650
I20260811 09:01:26.493227 13599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:26.511204 13599 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:26.513082 13599 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:26.522538 13599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e3a7c79c18c4577bdb3f487d94a3db4: Generated new TSK 0
I20260811 09:01:26.523591 13599 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:26.590575 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:26.936587 13615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:26.936956 13615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:26.937052 13615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:26.944126 13615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:26.944226 13615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:26.944335 13615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:26.953925 13615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:26.955103 13615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:26.956311 13615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:26.969408 13623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:26.969501 13621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:26.969457 13620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:26.975734 13615 server_base.cc:1034] running on GCE node
I20260811 09:01:26.976361 13615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:26.977473 13615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:26.978688 13615 hybrid_clock.cc:648] HybridClock initialized: now 1786438886978622 us; error 56 us; skew 500 ppm
I20260811 09:01:26.978981 13615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:26 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438886, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:26.982369 13615 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:26.983839 13615 webserver.cc:460] Webserver started at http://127.9.40.65:38707/ using document root <none> and password file <none>
I20260811 09:01:26.984512 13615 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:26.984611 13615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:26.984941 13615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:26.987922 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "dc3afe3a8d7949a8b8906bc3546f4165"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "affda3ea1c1f166eaa61eddcecaccd3c"
server_key_iv: "cc90f508fafe2ef62e990746075218bd"
server_key_version: "encryptionkey@0"
I20260811 09:01:26.988884 13615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "dc3afe3a8d7949a8b8906bc3546f4165"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "affda3ea1c1f166eaa61eddcecaccd3c"
server_key_iv: "cc90f508fafe2ef62e990746075218bd"
server_key_version: "encryptionkey@0"
I20260811 09:01:27.002835 13615 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.006s sys 0.007s
I20260811 09:01:27.007256 13631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.008569 13615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:27.008795 13615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "dc3afe3a8d7949a8b8906bc3546f4165"
format_stamp: "Formatted at 2026-08-11 09:01:26 on dist-test-slave-cbsf"
server_key: "affda3ea1c1f166eaa61eddcecaccd3c"
server_key_iv: "cc90f508fafe2ef62e990746075218bd"
server_key_version: "encryptionkey@0"
I20260811 09:01:27.009016 13615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:27.024040 13615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:27.027976 13615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:27.028260 13615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:27.029305 13615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:27.030565 13615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:27.030666 13615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.030809 13615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:27.030884 13615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.064394 13615 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33685
I20260811 09:01:27.064441 13758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33685 every 8 connection(s)
I20260811 09:01:27.065809 13615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:27.073246 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13615
I20260811 09:01:27.073482 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:27.074326 9377 external_mini_cluster.cc:1428] Setting key 85d789c036353c44804bc7f6c686e716
Aug 11 09:01:27 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438886, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:27.087796 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:27.068578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45339 (local address 127.9.40.126:36817)
0811 09:01:27.069090 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:27.069095 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:27.070001 (+ 906us) server_negotiation.cc:407] Connection header received
0811 09:01:27.071915 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:27.071920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:27.072013 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:27.072177 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:27.074190 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.075121 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.077276 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.077685 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.077784 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:27.081969 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:27.081974 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:27.081977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:27.082007 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:27.084664 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:27.085430 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:27.085436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:27.085439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:27.085569 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:27.086016 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:27.086020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:27.086022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:27.086274 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:27.086433 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:27.087117 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 09:01:27.087347 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":298,"threads_started":1}
I20260811 09:01:27.088765 13762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:27.069096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36817 (local address 127.9.40.65:45339)
0811 09:01:27.069805 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:27.069861 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:27.071599 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:27.072404 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:27.072412 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:27.073168 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:27.073987 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:27.074015 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.075295 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.075300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:27.077123 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:27.077155 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.077926 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.077937 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:27.078288 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:27.079024 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:27.079066 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:27.081734 (+ 2668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:27.084850 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:27.084859 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:27.084873 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:27.085301 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:27.085681 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:27.085686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:27.085689 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:27.085883 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:27.086405 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:27.086413 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:27.086755 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 09:01:27.087013 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 09:01:27.087549 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":286,"threads_started":1}
I20260811 09:01:27.090242 13759 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36817
I20260811 09:01:27.090586 13759 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:27.091513 13759 heartbeater.cc:511] Master 127.9.40.126:36817 requested a full tablet report, sending...
I20260811 09:01:27.093854 13546 ts_manager.cc:194] Registered new tserver with Master: dc3afe3a8d7949a8b8906bc3546f4165 (127.9.40.65:33685)
I20260811 09:01:27.095323 13546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:45339
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:27.125969 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:27.452719 13765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:27.452999 13765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:27.453191 13765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:27.461616 13765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:27.461762 13765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:27.461956 13765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:27.472086 13765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:27.473538 13765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:27.474853 13765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:27.488418 13771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:27.488603 13773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:27.489004 13770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:27.489168 13765 server_base.cc:1034] running on GCE node
I20260811 09:01:27.489835 13765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:27.490769 13765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:27.492003 13765 hybrid_clock.cc:648] HybridClock initialized: now 1786438887491946 us; error 46 us; skew 500 ppm
I20260811 09:01:27.492349 13765 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:27 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438887, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:27.495599 13765 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:27.497213 13765 webserver.cc:460] Webserver started at http://127.9.40.66:33435/ using document root <none> and password file <none>
I20260811 09:01:27.497925 13765 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:27.498059 13765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:27.498389 13765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:27.501475 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "c13dcdfd399e4bbbaa13349470fc81ce"
format_stamp: "Formatted at 2026-08-11 09:01:27 on dist-test-slave-cbsf"
server_key: "2a7dc2a541e2fc7103947330d2266c7b"
server_key_iv: "b0b6cc17e51a87bd2d995a02f83784a0"
server_key_version: "encryptionkey@0"
I20260811 09:01:27.502254 13765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "c13dcdfd399e4bbbaa13349470fc81ce"
format_stamp: "Formatted at 2026-08-11 09:01:27 on dist-test-slave-cbsf"
server_key: "2a7dc2a541e2fc7103947330d2266c7b"
server_key_iv: "b0b6cc17e51a87bd2d995a02f83784a0"
server_key_version: "encryptionkey@0"
I20260811 09:01:27.507344 13765 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:01:27.510851 13780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.512195 13765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:27.512377 13765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "c13dcdfd399e4bbbaa13349470fc81ce"
format_stamp: "Formatted at 2026-08-11 09:01:27 on dist-test-slave-cbsf"
server_key: "2a7dc2a541e2fc7103947330d2266c7b"
server_key_iv: "b0b6cc17e51a87bd2d995a02f83784a0"
server_key_version: "encryptionkey@0"
I20260811 09:01:27.512585 13765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:27.530998 13765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:27.535054 13765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:27.535354 13765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:27.536401 13765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:27.537839 13765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:27.537912 13765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.538103 13765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:27.538156 13765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:27.572360 13765 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39271
I20260811 09:01:27.572490 13914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39271 every 8 connection(s)
I20260811 09:01:27.573823 13765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:27.576934 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13765
I20260811 09:01:27.577126 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:27.577761 9377 external_mini_cluster.cc:1428] Setting key 0057e88f6bc8d65b29be591af80c4651
Aug 11 09:01:27 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438887, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:27.597201 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:27.576456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56953 (local address 127.9.40.126:36817)
0811 09:01:27.576754 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:27.576760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:27.578109 (+ 1349us) server_negotiation.cc:407] Connection header received
0811 09:01:27.580400 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:27.580404 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:27.580469 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:27.580550 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:27.582784 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.583496 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.586038 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.586475 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.586568 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:27.591124 (+ 4556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:27.591129 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:27.591132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:27.591164 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:27.593836 (+ 2672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:27.594797 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:27.594800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:27.594803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:27.594927 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:27.595398 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:27.595401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:27.595403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:27.595663 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:27.595771 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:27.596686 (+ 915us) server_negotiation.cc:299] Negotiation successful
0811 09:01:27.596929 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 09:01:27.598855 13918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:27.576949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36817 (local address 127.9.40.66:56953)
0811 09:01:27.577906 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:27.577974 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:27.580099 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:27.580865 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:27.580879 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:27.581733 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:27.582588 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:27.582615 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.583662 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.583670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:27.585894 (+ 2224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:27.585912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:27.586705 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:27.586715 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:27.587045 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:27.587787 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:27.587822 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:27.590870 (+ 3048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:27.593989 (+ 3119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:27.594001 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:27.594021 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:27.594674 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:27.595030 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:27.595038 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:27.595044 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:27.595284 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:27.595799 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:27.595810 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:27.596297 (+ 487us) client_negotiation.cc:769] Sending connection context
0811 09:01:27.596606 (+ 309us) client_negotiation.cc:240] Negotiation successful
0811 09:01:27.597299 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":377,"threads_started":1}
I20260811 09:01:27.600409 13915 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36817
I20260811 09:01:27.600798 13915 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:27.601758 13915 heartbeater.cc:511] Master 127.9.40.126:36817 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260811 09:01:27.603439 13546 ts_manager.cc:194] Registered new tserver with Master: c13dcdfd399e4bbbaa13349470fc81ce (127.9.40.66:39271)
I20260811 09:01:27.604369 13546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:56953
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:27.633822 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:27.963166 13921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:27.963472 13921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:27.963667 13921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:27.970728 13921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:27.970863 13921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:27.971022 13921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:27.980762 13921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:27.981972 13921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:27.983210 13921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:27.996344 13926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:27.996470 13927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:27.996905 13929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:27.997359 13921 server_base.cc:1034] running on GCE node
I20260811 09:01:27.997892 13921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:27.998770 13921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:28.000018 13921 hybrid_clock.cc:648] HybridClock initialized: now 1786438887999946 us; error 55 us; skew 500 ppm
I20260811 09:01:28.000339 13921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:27 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438887, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:28.003473 13921 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:28.005043 13921 webserver.cc:460] Webserver started at http://127.9.40.67:33821/ using document root <none> and password file <none>
I20260811 09:01:28.005681 13921 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:28.005827 13921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:28.006139 13921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:28.008975 13921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "289ac1a8a5d748738c6fec0858e6e671"
format_stamp: "Formatted at 2026-08-11 09:01:28 on dist-test-slave-cbsf"
server_key: "8855d0f2515ae73c3ed23fde48239410"
server_key_iv: "f38f4acf713420ce0f9144e91de8e5a8"
server_key_version: "encryptionkey@0"
I20260811 09:01:28.009737 13921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "289ac1a8a5d748738c6fec0858e6e671"
format_stamp: "Formatted at 2026-08-11 09:01:28 on dist-test-slave-cbsf"
server_key: "8855d0f2515ae73c3ed23fde48239410"
server_key_iv: "f38f4acf713420ce0f9144e91de8e5a8"
server_key_version: "encryptionkey@0"
I20260811 09:01:28.014881 13921 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:01:28.018589 13937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:28.019896 13921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:28.020097 13921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "289ac1a8a5d748738c6fec0858e6e671"
format_stamp: "Formatted at 2026-08-11 09:01:28 on dist-test-slave-cbsf"
server_key: "8855d0f2515ae73c3ed23fde48239410"
server_key_iv: "f38f4acf713420ce0f9144e91de8e5a8"
server_key_version: "encryptionkey@0"
I20260811 09:01:28.020305 13921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:28.044368 13921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:28.048838 13921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:28.049196 13921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:28.050263 13921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:28.051671 13921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:28.051784 13921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:28.051940 13921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:28.052027 13921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:28.088142 13921 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:40067
I20260811 09:01:28.088213 14074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:40067 every 8 connection(s)
I20260811 09:01:28.089708 13921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:28.095795 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 13921
I20260811 09:01:28.095979 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:28.096567 9377 external_mini_cluster.cc:1428] Setting key a27ffad87b70cd1614f815f46209be3a
I20260811 09:01:28.098954 13759 heartbeater.cc:503] Master 127.9.40.126:36817 was elected leader, sending a full tablet report...
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438887, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:28.110499 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.092325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45169 (local address 127.9.40.126:36817)
0811 09:01:28.092499 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.092503 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.093766 (+ 1263us) server_negotiation.cc:407] Connection header received
0811 09:01:28.095709 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.095714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.095815 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.095961 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.098075 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.099216 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.101294 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.101709 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.101842 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.105129 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.105133 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.105136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.105166 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.107579 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.108219 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.108223 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.108228 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.108355 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.108792 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.108797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.108800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.109067 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.109229 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.109884 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.110127 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:01:28.111459 14077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.092920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36817 (local address 127.9.40.67:45169)
0811 09:01:28.093614 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:28.093652 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:28.095356 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:28.096206 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:28.096217 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:28.096965 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:28.097861 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:28.097885 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.099343 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.099348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:28.101149 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:28.101162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.101928 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.101935 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:28.102182 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.102774 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:28.102793 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:28.104927 (+ 2134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:28.107706 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:28.107714 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:28.107726 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:28.108102 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:28.108457 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:28.108462 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:28.108465 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:28.108638 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:28.109205 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:28.109212 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:28.109503 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 09:01:28.109778 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 09:01:28.110249 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":257,"threads_started":1}
I20260811 09:01:28.112967 14075 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36817
I20260811 09:01:28.113237 14075 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:28.113945 14075 heartbeater.cc:511] Master 127.9.40.126:36817 requested a full tablet report, sending...
I20260811 09:01:28.115370 13546 ts_manager.cc:194] Registered new tserver with Master: 289ac1a8a5d748738c6fec0858e6e671 (127.9.40.67:40067)
I20260811 09:01:28.116183 13546 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:45169
I20260811 09:01:28.124871 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:28.178829 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.163124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48264 (local address 127.9.40.65:33685)
0811 09:01:28.163806 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.163814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.163843 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:28.163905 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.163912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.164128 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.164260 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.165795 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.166870 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.168102 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.168688 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.168853 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.173365 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.173375 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.173390 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.173426 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.176242 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.176938 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.176945 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.176948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.177066 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.177422 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.177425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.177427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.177757 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.177981 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.178251 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.178471 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":332,"threads_started":1}
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:28.196616 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.183708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.9.40.126:36817)
0811 09:01:28.183916 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.183921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.183997 (+ 76us) server_negotiation.cc:407] Connection header received
0811 09:01:28.184126 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.184130 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.184191 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.184268 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.185647 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.186327 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.187830 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.188274 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.188370 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.192020 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.192025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.192028 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.192060 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.194435 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.194995 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.194998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.195000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.195122 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.195419 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.195422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.195424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.195722 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.195896 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.196120 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.196273 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 09:01:28.197362 13546 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39410
I20260811 09:01:28.198316 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13615
I20260811 09:01:28.209859 14083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.199131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.9.40.65:33685)
0811 09:01:28.199355 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.199361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.199518 (+ 157us) server_negotiation.cc:407] Connection header received
0811 09:01:28.199618 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.199621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.199677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.199750 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.200901 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.201505 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.202533 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.202911 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.202994 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.205265 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.205270 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.205273 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.205300 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.207735 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.208255 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.208259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.208261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.208378 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.208732 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.208739 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.208746 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.208997 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.209159 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.209392 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.209539 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:01:28.210465 13754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:28.294332 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13615
I20260811 09:01:28.313962 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13765
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:28.330770 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.314946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36390 (local address 127.9.40.66:39271)
0811 09:01:28.315444 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.315452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.315481 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:28.315550 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.315556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.315774 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.315954 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.317324 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.318434 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.319684 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.320243 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.320390 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.325050 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.325064 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.325081 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.325116 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.328115 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.328868 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.328875 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.328877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.329022 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.329406 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.329409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.329411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.329747 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.330011 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.330260 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.330463 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":248,"threads_started":1}
I20260811 09:01:28.332084 13910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:28.414819 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13765
I20260811 09:01:28.439975 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13921
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[13495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:28.457069 14091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:28.441236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34882 (local address 127.9.40.67:40067)
0811 09:01:28.441960 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:28.441967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:28.441999 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:01:28.442064 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:28.442071 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:28.442300 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:28.442427 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:28.443731 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.444930 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.446158 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:28.446787 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:28.446894 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:28.451426 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:28.451438 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:28.451484 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:28.451527 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:28.454550 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.455167 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.455174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.455176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.455294 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:28.455598 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:28.455601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:28.455603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:28.455953 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:28.456193 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:28.456392 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 09:01:28.456611 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":396,"threads_started":1}
I20260811 09:01:28.458480 14070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:28.543293 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13921
I20260811 09:01:28.562593 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 13509
I20260811 09:01:28.563839 13581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:28.668941 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 13509
2026-08-11T09:01:28Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4079 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[14097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[14097](info): set up 2 sockets
Aug 11 09:01:28 dist-test-slave-cbsf krb5kdc[14097](info): commencing operation
krb5kdc: starting...
W20260811 09:01:29.856443 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:30 dist-test-slave-cbsf krb5kdc[14097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:30.085631 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36299
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:30.420181 14112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:30.420575 14112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:30.420789 14112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:30.428349 14112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:30.428498 14112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:30.428607 14112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:30.428735 14112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:30.428833 14112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:30.438905 14112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:30.440320 14112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:30.441679 14112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:30.453614 14121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:30.453704 14119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:30.453668 14118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:30.454916 14112 server_base.cc:1034] running on GCE node
I20260811 09:01:30.455718 14112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:30.457300 14112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:30.458539 14112 hybrid_clock.cc:648] HybridClock initialized: now 1786438890458504 us; error 51 us; skew 500 ppm
I20260811 09:01:30.458901 14112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:30 dist-test-slave-cbsf krb5kdc[14097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:30.462105 14112 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:30.463367 14112 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:30.463506 14112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:30.463850 14112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:30.466952 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "f99881cf5da5438782aefc6f61afc77d"
format_stamp: "Formatted at 2026-08-11 09:01:30 on dist-test-slave-cbsf"
server_key: "38d9295d37d123002888ab920e2ecf64"
server_key_iv: "8d71c578f1f0a2360ec18dd208d8e3e4"
server_key_version: "encryptionkey@0"
I20260811 09:01:30.467773 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "f99881cf5da5438782aefc6f61afc77d"
format_stamp: "Formatted at 2026-08-11 09:01:30 on dist-test-slave-cbsf"
server_key: "38d9295d37d123002888ab920e2ecf64"
server_key_iv: "8d71c578f1f0a2360ec18dd208d8e3e4"
server_key_version: "encryptionkey@0"
I20260811 09:01:30.474191 14112 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260811 09:01:30.477778 14128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:30.479122 14112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:01:30.479326 14112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "f99881cf5da5438782aefc6f61afc77d"
format_stamp: "Formatted at 2026-08-11 09:01:30 on dist-test-slave-cbsf"
server_key: "38d9295d37d123002888ab920e2ecf64"
server_key_iv: "8d71c578f1f0a2360ec18dd208d8e3e4"
server_key_version: "encryptionkey@0"
I20260811 09:01:30.479565 14112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:30.503295 14112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:30.507725 14112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:30.508067 14112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:30.528373 14112 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36299
I20260811 09:01:30.528410 14191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36299 every 8 connection(s)
I20260811 09:01:30.529402 14112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:30.533324 14192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:30.536983 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14112
I20260811 09:01:30.537241 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:30.538125 9377 external_mini_cluster.cc:1428] Setting key 12f303771dfb092a02a281b82404e54e
I20260811 09:01:30.547101 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: Bootstrap starting.
I20260811 09:01:30.551100 14192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:30.552443 14192 log.cc:824] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:01:30 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:30.555544 14192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: No bootstrap required, opened a new log
I20260811 09:01:30.562050 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:30.540330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.9.40.126:36299)
0811 09:01:30.541181 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:30.541193 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:30.541233 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:01:30.542855 (+ 1622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:30.542878 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:30.543254 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:30.543583 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:30.545514 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:30.546798 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:30.548675 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:30.549428 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:30.549599 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:30.554615 (+ 5016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:30.554622 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:30.554636 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:30.554668 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:30.557905 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:30.558696 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:30.558702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:30.558708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:30.558883 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:30.559313 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:30.559316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:30.559321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:30.559831 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:30.560063 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:30.560550 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 09:01:30.561013 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":428,"threads_started":1}
I20260811 09:01:30.564670 14192 raft_consensus.cc:348] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } }
I20260811 09:01:30.565210 14192 raft_consensus.cc:374] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:30.565320 14192 raft_consensus.cc:729] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99881cf5da5438782aefc6f61afc77d, State: Initialized, Role: FOLLOWER
I20260811 09:01:30.565868 14192 consensus_queue.cc:260] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } }
I20260811 09:01:30.566145 14192 raft_consensus.cc:388] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:30.566294 14192 raft_consensus.cc:482] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:30.566468 14192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:30.568672 14192 raft_consensus.cc:504] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } }
I20260811 09:01:30.569240 14192 leader_election.cc:302] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f99881cf5da5438782aefc6f61afc77d; no voters:
I20260811 09:01:30.569955 14192 leader_election.cc:288] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:30.570118 14197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:30.571255 14197 raft_consensus.cc:686] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [term 1 LEADER]: Becoming Leader. State: Replica: f99881cf5da5438782aefc6f61afc77d, State: Running, Role: LEADER
I20260811 09:01:30.571795 14197 consensus_queue.cc:237] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } }
I20260811 09:01:30.572183 14192 sys_catalog.cc:564] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:30.575011 14200 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f99881cf5da5438782aefc6f61afc77d. Latest consensus state: current_term: 1 leader_uuid: "f99881cf5da5438782aefc6f61afc77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } } }
I20260811 09:01:30.575424 14200 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:30.575709 14199 sys_catalog.cc:455] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f99881cf5da5438782aefc6f61afc77d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f99881cf5da5438782aefc6f61afc77d" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36299 } } }
I20260811 09:01:30.576097 14199 sys_catalog.cc:458] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:30.577343 14208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:30.583494 14208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:30.591813 14208 catalog_manager.cc:1347] Generated new cluster ID: be980096b3f745118b0260f6d2c80ed6
I20260811 09:01:30.591933 14208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:30.601784 14208 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:30.603559 14208 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:30.611819 14208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: Generated new TSK 0
I20260811 09:01:30.612771 14208 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:30.668957 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:31.002564 14221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:31.002815 14221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:31.002911 14221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:31.009929 14221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:31.010031 14221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:31.010147 14221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:31.020865 14221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:31.022174 14221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:31.023530 14221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:31.037271 14227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:31.037221 14229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:31.037313 14232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:31.038960 14221 server_base.cc:1034] running on GCE node
I20260811 09:01:31.039661 14221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:31.040657 14221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:31.041934 14221 hybrid_clock.cc:648] HybridClock initialized: now 1786438891041895 us; error 33 us; skew 500 ppm
I20260811 09:01:31.042277 14221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:31 dist-test-slave-cbsf krb5kdc[14097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438891, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:31.045574 14221 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:31.046789 14221 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:31.046926 14221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:31.047252 14221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:31.050169 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "fe1bf8a344d745b4810b1258d47e89c5"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "bc6682ad93bc7b99f5ec27b8ef6c62d9"
server_key_iv: "4a9496a99551fe4f39e880db3781a6ae"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.050995 14221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "fe1bf8a344d745b4810b1258d47e89c5"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "bc6682ad93bc7b99f5ec27b8ef6c62d9"
server_key_iv: "4a9496a99551fe4f39e880db3781a6ae"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.055919 14221 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 09:01:31.059376 14242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.060755 14221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:31.060920 14221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "fe1bf8a344d745b4810b1258d47e89c5"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "bc6682ad93bc7b99f5ec27b8ef6c62d9"
server_key_iv: "4a9496a99551fe4f39e880db3781a6ae"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.061161 14221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:31.073933 14221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:31.078130 14221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:31.078454 14221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:31.079365 14221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:31.080875 14221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:31.081002 14221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.081139 14221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:31.081231 14221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.117169 14221 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:41741
I20260811 09:01:31.117219 14370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:41741 every 8 connection(s)
I20260811 09:01:31.118163 14221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:31.120837 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14221
I20260811 09:01:31.121050 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:31.121943 9377 external_mini_cluster.cc:1428] Setting key 964ca887b99651b3dfc60d92c54648f3
Aug 11 09:01:31 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438891, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:31.138749 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:31.120895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36887 (local address 127.9.40.126:36299)
0811 09:01:31.121449 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:31.121454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:31.122158 (+ 704us) server_negotiation.cc:407] Connection header received
0811 09:01:31.124291 (+ 2133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:31.124297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:31.124391 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:31.124517 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:31.126179 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.126922 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.128600 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.129333 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.129481 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:31.133097 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:31.133101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:31.133104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:31.133133 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:31.135683 (+ 2550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:31.136442 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:31.136449 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:31.136458 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:31.136581 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:31.137046 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:31.137050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:31.137056 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:31.137324 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:31.137483 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:31.138143 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 09:01:31.138380 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":297,"threads_started":1}
I20260811 09:01:31.139624 14374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:31.121209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36299 (local address 127.9.40.65:36887)
0811 09:01:31.121986 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:31.122033 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:31.124009 (+ 1976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:31.124811 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:31.124820 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:31.125350 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:31.126008 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:31.126027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.127103 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.127108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:31.128400 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:31.128413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.129589 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.129595 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:31.129787 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:31.130521 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:31.130550 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:31.132900 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:31.135850 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:31.135859 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:31.135875 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:31.136309 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:31.136740 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:31.136745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:31.136749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:31.136923 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:31.137455 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:31.137463 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:31.137783 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 09:01:31.138035 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 09:01:31.138546 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":333,"threads_started":1}
I20260811 09:01:31.141259 14371 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36299
I20260811 09:01:31.141592 14371 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:31.142441 14371 heartbeater.cc:511] Master 127.9.40.126:36299 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:01:31.144992 14149 ts_manager.cc:194] Registered new tserver with Master: fe1bf8a344d745b4810b1258d47e89c5 (127.9.40.65:41741)
I20260811 09:01:31.146499 14149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:31.171190 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:31.494771 14377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:31.495038 14377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:31.495138 14377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:31.502417 14377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:31.502554 14377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:31.502671 14377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:31.512338 14377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:31.513621 14377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:31.514909 14377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:31.528820 14383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:31.528836 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:31.528888 14382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:31.534243 14377 server_base.cc:1034] running on GCE node
I20260811 09:01:31.534799 14377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:31.535693 14377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:31.536921 14377 hybrid_clock.cc:648] HybridClock initialized: now 1786438891536875 us; error 56 us; skew 500 ppm
I20260811 09:01:31.537256 14377 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:31 dist-test-slave-cbsf krb5kdc[14097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438891, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:31.545276 14377 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:31.546438 14377 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:31.546525 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:31.546777 14377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:31.549790 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "d7791e7920584c27a57f89030b70c14f"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "3e70e93935a3b2c626613989fb00737a"
server_key_iv: "c9cdcd65f7030c5d53f16c90d1e28047"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.550537 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "d7791e7920584c27a57f89030b70c14f"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "3e70e93935a3b2c626613989fb00737a"
server_key_iv: "c9cdcd65f7030c5d53f16c90d1e28047"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.555362 14377 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:01:31.558627 14393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.559923 14377 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:01:31.560093 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "d7791e7920584c27a57f89030b70c14f"
format_stamp: "Formatted at 2026-08-11 09:01:31 on dist-test-slave-cbsf"
server_key: "3e70e93935a3b2c626613989fb00737a"
server_key_iv: "c9cdcd65f7030c5d53f16c90d1e28047"
server_key_version: "encryptionkey@0"
I20260811 09:01:31.560309 14377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:31.584039 14377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:31.588310 14377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:31.588608 14377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:31.589689 14377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:31.591038 14377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:31.591178 14377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.591317 14377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:31.591411 14377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:31.625943 14377 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:34309
I20260811 09:01:31.626037 14522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:34309 every 8 connection(s)
I20260811 09:01:31.626899 14377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:31.630824 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14377
I20260811 09:01:31.631019 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:31.631665 9377 external_mini_cluster.cc:1428] Setting key 145ac3131f8998ec0c4b13a3d12a5950
Aug 11 09:01:31 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438891, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:31.649005 14373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:31.629653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:60139 (local address 127.9.40.126:36299)
0811 09:01:31.629839 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:31.629845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:31.631172 (+ 1327us) server_negotiation.cc:407] Connection header received
0811 09:01:31.633640 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:31.633649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:31.633752 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:31.633840 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:31.635339 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.636583 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.638220 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.639025 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.639188 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:31.642689 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:31.642703 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:31.642708 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:31.642742 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:31.645554 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:31.646529 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:31.646533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:31.646536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:31.646684 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:31.647153 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:31.647156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:31.647158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:31.647440 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:31.647560 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:31.648466 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 09:01:31.648665 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:01:31.650393 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:31.630130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36299 (local address 127.9.40.66:60139)
0811 09:01:31.630951 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:31.631011 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:31.633318 (+ 2307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:31.634090 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:31.634098 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:31.634607 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:31.635146 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:31.635163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.636784 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.636788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:31.638036 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:31.638049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:31.639234 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:31.639238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:31.639419 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:31.640173 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:31.640207 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:31.642443 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:31.645713 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:31.645724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:31.645743 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:31.646391 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:31.646794 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:31.646801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:31.646808 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:31.647034 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:31.647579 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:31.647589 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:31.648043 (+ 454us) client_negotiation.cc:769] Sending connection context
0811 09:01:31.648346 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 09:01:31.648930 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":415,"threads_started":1}
I20260811 09:01:31.652000 14523 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36299
I20260811 09:01:31.652417 14523 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:31.653175 14523 heartbeater.cc:511] Master 127.9.40.126:36299 requested a full tablet report, sending...
I20260811 09:01:31.654742 14149 ts_manager.cc:194] Registered new tserver with Master: d7791e7920584c27a57f89030b70c14f (127.9.40.66:34309)
I20260811 09:01:31.655969 14149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:60139
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:31.684060 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:32.007166 14529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:32.007433 14529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:32.007594 14529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:32.014882 14529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:32.014981 14529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:32.015208 14529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:32.025303 14529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36299
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:32.026602 14529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:32.028046 14529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:32.041533 14535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:32.041618 14538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:32.041570 14536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:32.043221 14529 server_base.cc:1034] running on GCE node
I20260811 09:01:32.043887 14529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:32.044921 14529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:32.046130 14529 hybrid_clock.cc:648] HybridClock initialized: now 1786438892046091 us; error 34 us; skew 500 ppm
I20260811 09:01:32.046458 14529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:32 dist-test-slave-cbsf krb5kdc[14097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:32.049710 14529 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:32.050810 14529 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:32.050904 14529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:32.051155 14529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:32.054070 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "a6c8ee1d167645c3bf97ff655ade436f"
format_stamp: "Formatted at 2026-08-11 09:01:32 on dist-test-slave-cbsf"
server_key: "17e7cee85fcfe22c0ec19bc34c402d50"
server_key_iv: "95da602d372676f3836e3a3b0941ac6e"
server_key_version: "encryptionkey@0"
I20260811 09:01:32.054850 14529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "a6c8ee1d167645c3bf97ff655ade436f"
format_stamp: "Formatted at 2026-08-11 09:01:32 on dist-test-slave-cbsf"
server_key: "17e7cee85fcfe22c0ec19bc34c402d50"
server_key_iv: "95da602d372676f3836e3a3b0941ac6e"
server_key_version: "encryptionkey@0"
I20260811 09:01:32.061301 14529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 09:01:32.069254 14546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:32.071131 14529 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.005s
I20260811 09:01:32.071329 14529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "a6c8ee1d167645c3bf97ff655ade436f"
format_stamp: "Formatted at 2026-08-11 09:01:32 on dist-test-slave-cbsf"
server_key: "17e7cee85fcfe22c0ec19bc34c402d50"
server_key_iv: "95da602d372676f3836e3a3b0941ac6e"
server_key_version: "encryptionkey@0"
I20260811 09:01:32.071609 14529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:32.109825 14529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:32.114743 14529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:32.115054 14529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:32.116074 14529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:32.117882 14529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:32.118005 14529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:32.118126 14529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:32.118211 14529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:32.149515 14371 heartbeater.cc:503] Master 127.9.40.126:36299 was elected leader, sending a full tablet report...
I20260811 09:01:32.158519 14529 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35771
I20260811 09:01:32.158607 14673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35771 every 8 connection(s)
I20260811 09:01:32.159379 14529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:32.166054 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14529
I20260811 09:01:32.166239 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:32.166824 9377 external_mini_cluster.cc:1428] Setting key 3dcde4c275e5c80624ebb1e9666a077a
Aug 11 09:01:32 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:32.183899 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.162106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:51973 (local address 127.9.40.126:36299)
0811 09:01:32.162573 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.162581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.163563 (+ 982us) server_negotiation.cc:407] Connection header received
0811 09:01:32.166534 (+ 2971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.166540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.166623 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.166748 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:32.168948 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.170102 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.172670 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.173188 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.173280 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.178157 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:32.178163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:32.178167 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:32.178206 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:32.180951 (+ 2745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.181640 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.181647 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.181655 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.181772 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.182185 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.182189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.182198 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.182444 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:32.182614 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.183285 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.183504 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":213,"threads_started":1}
I20260811 09:01:32.184876 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.162602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36299 (local address 127.9.40.67:51973)
0811 09:01:32.163396 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.163453 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.166202 (+ 2749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.167004 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.167028 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.167854 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:32.168718 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.168771 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.170280 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.170287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.172535 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.172552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.173411 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.173419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.173728 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.174562 (+ 834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:32.174598 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:32.177922 (+ 3324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:32.181096 (+ 3174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:32.181105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:32.181118 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:32.181507 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:32.181880 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:32.181885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:32.181889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:32.182066 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:32.182577 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:32.182584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:32.182913 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.183187 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.183661 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":300,"threads_started":1}
I20260811 09:01:32.186138 14674 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36299
I20260811 09:01:32.186448 14674 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:32.187069 14674 heartbeater.cc:511] Master 127.9.40.126:36299 requested a full tablet report, sending...
I20260811 09:01:32.188465 14149 ts_manager.cc:194] Registered new tserver with Master: a6c8ee1d167645c3bf97ff655ade436f (127.9.40.67:35771)
I20260811 09:01:32.189313 14149 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:51973
I20260811 09:01:32.194329 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:01:32.210632 14676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.198717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.9.40.126:36299)
0811 09:01:32.198880 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.198888 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.199333 (+ 445us) server_negotiation.cc:407] Connection header received
0811 09:01:32.199525 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.199531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.199607 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.199706 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:32.201215 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.201885 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.203108 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.203561 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.203695 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.205833 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:32.205839 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:32.205843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:32.205879 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:32.207986 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.208648 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.208653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.208656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.208855 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.209212 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.209217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.209220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.209521 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:32.209799 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.210117 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.210336 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:01:32.214520 14149 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38520:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:32.221318 14149 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:32.243755 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.232765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59006 (local address 127.9.40.66:34309)
0811 09:01:32.233334 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.233345 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.234054 (+ 709us) server_negotiation.cc:407] Connection header received
0811 09:01:32.234410 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.234423 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.234701 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.234919 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:32.237218 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.238400 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.241133 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.242504 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.242610 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.242745 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.243219 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.243417 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":341,"threads_started":1}
I20260811 09:01:32.243773 14691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.232514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:34309 (local address 127.0.0.1:59006)
0811 09:01:32.233773 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.233812 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.234100 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.235248 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.235254 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.236317 (+ 1063us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:32.236925 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.236943 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.238557 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.238568 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.240961 (+ 2393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.240976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.242847 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.242851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.242980 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.243009 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.243165 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.243369 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":3,"thread_start_us":920,"threads_started":1}
I20260811 09:01:32.244868 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.233909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41118 (local address 127.9.40.67:35771)
0811 09:01:32.234670 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.234680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.235226 (+ 546us) server_negotiation.cc:407] Connection header received
0811 09:01:32.235318 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.235326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.235530 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.235726 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:32.238551 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.240104 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.242198 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.243445 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.243566 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.243785 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.244312 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.244481 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":434,"threads_started":1}
I20260811 09:01:32.245705 14695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.235026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41741 (local address 127.0.0.1:50978)
0811 09:01:32.236349 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.236373 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.236498 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.238181 (+ 1683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.238186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.238214 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:32.238687 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.238697 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.240032 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.240039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.242404 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.242419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.244233 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.244243 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.244408 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.244441 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.244502 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.244642 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"mutex_wait_us":78,"thread_start_us":1067,"threads_started":1}
I20260811 09:01:32.245779 14693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.232832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35771 (local address 127.0.0.1:41118)
0811 09:01:32.234944 (+ 2112us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.234968 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.235102 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.235716 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.235721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.237813 (+ 2092us) spinlock_profiling.cc:243] Waited 1.05 ms on lock 0x6120000231a8. stack: 00007f738796e66e 00007f738796e430 00007f73869f55ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f73879640fd 00007f7387963d1e 00007f739a86d3cd 00007f739a86d36c 00007f739a839b3b 00007f738e3758e9 00007f738e371e45 00007f738e3f9dca 00007f738e3f7f68 00007f738e42b82f
0811 09:01:32.237836 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:32.238263 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.238280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.240392 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.240398 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.242045 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.242056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.243899 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.243905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.244091 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.244134 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.245170 (+ 1036us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.245299 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1974,"spinlock_wait_cycles":2420096,"thread_start_us":1834,"threads_started":1}
I20260811 09:01:32.246254 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.235589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.9.40.65:41741)
0811 09:01:32.237541 (+ 1952us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.237550 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.237588 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:01:32.237701 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.237711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.237956 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.238170 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:32.238842 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.239863 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.242583 (+ 2720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.243996 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.245391 (+ 1395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.245579 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.245630 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.245966 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1713,"thread_start_us":1539,"threads_started":1}
I20260811 09:01:32.247985 14448 tablet_service.cc:1467] Processing CreateTablet for tablet 156a246743c24cd8a36844957700881a (DEFAULT_TABLE table=test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:32.248764 14600 tablet_service.cc:1467] Processing CreateTablet for tablet 156a246743c24cd8a36844957700881a (DEFAULT_TABLE table=test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:32.249514 14448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 156a246743c24cd8a36844957700881a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:32.249552 14293 tablet_service.cc:1467] Processing CreateTablet for tablet 156a246743c24cd8a36844957700881a (DEFAULT_TABLE table=test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:32.250113 14600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 156a246743c24cd8a36844957700881a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:32.250860 14293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 156a246743c24cd8a36844957700881a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:32.260767 14697 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Bootstrap starting.
I20260811 09:01:32.265161 14697 tablet_bootstrap.cc:654] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:32.265668 14698 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Bootstrap starting.
I20260811 09:01:32.266691 14697 log.cc:824] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:32.267131 14699 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Bootstrap starting.
I20260811 09:01:32.269333 14698 tablet_bootstrap.cc:654] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:32.270182 14699 tablet_bootstrap.cc:654] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:32.270627 14698 log.cc:824] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:32.270954 14697 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: No bootstrap required, opened a new log
I20260811 09:01:32.271375 14699 log.cc:824] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:32.271849 14697 ts_tablet_manager.cc:1397] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260811 09:01:32.273929 14698 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: No bootstrap required, opened a new log
I20260811 09:01:32.274201 14698 ts_tablet_manager.cc:1397] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:01:32.278825 14699 tablet_bootstrap.cc:492] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: No bootstrap required, opened a new log
I20260811 09:01:32.279098 14699 ts_tablet_manager.cc:1397] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260811 09:01:32.283046 14697 raft_consensus.cc:348] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.283496 14697 raft_consensus.cc:374] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:32.283595 14697 raft_consensus.cc:729] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6c8ee1d167645c3bf97ff655ade436f, State: Initialized, Role: FOLLOWER
I20260811 09:01:32.284202 14697 consensus_queue.cc:260] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.284967 14698 raft_consensus.cc:348] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.285480 14698 raft_consensus.cc:374] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:32.285597 14698 raft_consensus.cc:729] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7791e7920584c27a57f89030b70c14f, State: Initialized, Role: FOLLOWER
I20260811 09:01:32.286288 14698 consensus_queue.cc:260] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.287508 14523 heartbeater.cc:503] Master 127.9.40.126:36299 was elected leader, sending a full tablet report...
I20260811 09:01:32.287652 14699 raft_consensus.cc:348] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.287997 14699 raft_consensus.cc:374] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:32.288113 14699 raft_consensus.cc:729] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1bf8a344d745b4810b1258d47e89c5, State: Initialized, Role: FOLLOWER
I20260811 09:01:32.288265 14698 ts_tablet_manager.cc:1428] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 09:01:32.288655 14699 consensus_queue.cc:260] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.289363 14674 heartbeater.cc:503] Master 127.9.40.126:36299 was elected leader, sending a full tablet report...
I20260811 09:01:32.290284 14697 ts_tablet_manager.cc:1428] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Time spent starting tablet: real 0.018s user 0.012s sys 0.004s
I20260811 09:01:32.292361 14699 ts_tablet_manager.cc:1428] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260811 09:01:32.370674 14372 tablet.cc:2369] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:32.377653 14704 raft_consensus.cc:482] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:32.378039 14704 raft_consensus.cc:504] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
W20260811 09:01:32.378801 14524 tablet.cc:2369] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:32.379971 14704 leader_election.cc:288] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7791e7920584c27a57f89030b70c14f (127.9.40.66:34309), fe1bf8a344d745b4810b1258d47e89c5 (127.9.40.65:41741)
I20260811 09:01:32.386322 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.380177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:34309 (local address 127.9.40.67:34891)
0811 09:01:32.380537 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.380558 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.380738 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.381103 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.381109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.381138 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:32.381334 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.381344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.382348 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.382359 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.383913 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.383927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.385645 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.385649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.385843 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.385870 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.385942 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.386026 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":34}
I20260811 09:01:32.386340 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.380355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34891 (local address 127.9.40.66:34309)
0811 09:01:32.380653 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.380659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.380679 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:01:32.380895 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.380899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.380959 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.381055 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:32.381476 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.382209 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.384092 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.385468 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.385625 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.385732 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.386005 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.386156 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260811 09:01:32.387387 14474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "156a246743c24cd8a36844957700881a" candidate_uuid: "a6c8ee1d167645c3bf97ff655ade436f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7791e7920584c27a57f89030b70c14f" is_pre_election: true
I20260811 09:01:32.388078 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.380455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34551 (local address 127.9.40.65:41741)
0811 09:01:32.380669 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.380678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.381476 (+ 798us) server_negotiation.cc:407] Connection header received
0811 09:01:32.381701 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.381705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.381768 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.381869 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:32.382444 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.383367 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.386119 (+ 2752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.387162 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.387259 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.387324 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.387834 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.387923 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:01:32.388001 14474 raft_consensus.cc:2454] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c8ee1d167645c3bf97ff655ade436f in term 0.
I20260811 09:01:32.388278 14707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.380438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41741 (local address 127.9.40.67:34551)
0811 09:01:32.381351 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:32.381367 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:32.381542 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:32.381897 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:32.381901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:32.381927 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:32.382256 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.382264 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.383524 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.383530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.385956 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:32.385972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.387364 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.387373 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:32.387613 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.387654 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 09:01:32.387756 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:01:32.387880 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":1,"thread_start_us":540,"threads_started":1}
I20260811 09:01:32.388765 14549 leader_election.cc:302] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c8ee1d167645c3bf97ff655ade436f, d7791e7920584c27a57f89030b70c14f; no voters:
I20260811 09:01:32.389258 14704 raft_consensus.cc:2781] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:32.389500 14704 raft_consensus.cc:482] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:32.389637 14704 raft_consensus.cc:3037] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:32.389551 14318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "156a246743c24cd8a36844957700881a" candidate_uuid: "a6c8ee1d167645c3bf97ff655ade436f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1bf8a344d745b4810b1258d47e89c5" is_pre_election: true
I20260811 09:01:32.390137 14318 raft_consensus.cc:2454] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a6c8ee1d167645c3bf97ff655ade436f in term 0.
I20260811 09:01:32.392338 14704 raft_consensus.cc:504] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.393035 14704 leader_election.cc:288] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [CANDIDATE]: Term 1 election: Requested vote from peers d7791e7920584c27a57f89030b70c14f (127.9.40.66:34309), fe1bf8a344d745b4810b1258d47e89c5 (127.9.40.65:41741)
I20260811 09:01:32.393359 14474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "156a246743c24cd8a36844957700881a" candidate_uuid: "a6c8ee1d167645c3bf97ff655ade436f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7791e7920584c27a57f89030b70c14f"
I20260811 09:01:32.393545 14474 raft_consensus.cc:3037] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:32.393745 14318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "156a246743c24cd8a36844957700881a" candidate_uuid: "a6c8ee1d167645c3bf97ff655ade436f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe1bf8a344d745b4810b1258d47e89c5"
I20260811 09:01:32.393977 14318 raft_consensus.cc:3037] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:32.396013 14474 raft_consensus.cc:2454] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c8ee1d167645c3bf97ff655ade436f in term 1.
I20260811 09:01:32.396358 14318 raft_consensus.cc:2454] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a6c8ee1d167645c3bf97ff655ade436f in term 1.
I20260811 09:01:32.396390 14549 leader_election.cc:302] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6c8ee1d167645c3bf97ff655ade436f, d7791e7920584c27a57f89030b70c14f; no voters:
I20260811 09:01:32.396852 14704 raft_consensus.cc:2781] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:32.397367 14704 raft_consensus.cc:686] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 1 LEADER]: Becoming Leader. State: Replica: a6c8ee1d167645c3bf97ff655ade436f, State: Running, Role: LEADER
I20260811 09:01:32.397966 14704 consensus_queue.cc:237] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } }
I20260811 09:01:32.402568 14147 catalog_manager.cc:5495] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f reported cstate change: term changed from 0 to 1, leader changed from <none> to a6c8ee1d167645c3bf97ff655ade436f (127.9.40.67). New cstate: current_term: 1 leader_uuid: "a6c8ee1d167645c3bf97ff655ade436f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a6c8ee1d167645c3bf97ff655ade436f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 } health_report { overall_health: UNKNOWN } } }
W20260811 09:01:32.410952 14675 tablet.cc:2369] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:32.434084 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.428836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.9.40.67:35771)
0811 09:01:32.429058 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.429064 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.429082 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:01:32.429228 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.429232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.429305 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.429424 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:32.429783 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.430498 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.431971 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.432373 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.432465 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.433019 (+ 554us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:32.433502 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:32.433618 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.433703 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.433811 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:01:32.442068 14318 raft_consensus.cc:1261] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 1 FOLLOWER]: Refusing update from remote peer a6c8ee1d167645c3bf97ff655ade436f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:32.442312 14474 raft_consensus.cc:1261] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 1 FOLLOWER]: Refusing update from remote peer a6c8ee1d167645c3bf97ff655ade436f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:32.443086 14708 consensus_queue.cc:1035] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe1bf8a344d745b4810b1258d47e89c5" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:32.443579 14704 consensus_queue.cc:1035] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7791e7920584c27a57f89030b70c14f" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:32.454614 14711 mvcc.cc:205] Tried to move back new op lower bound from 7317253703433728000 to 7317253703266414592. Current Snapshot: MvccSnapshot[applied={T|T < 7317253703433728000}]
I20260811 09:01:32.470417 14148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:32.470701 14148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38520:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:32.474666 14148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f99881cf5da5438782aefc6f61afc77d: Sending DeleteTablet for 3 replicas of tablet 156a246743c24cd8a36844957700881a
I20260811 09:01:32.475879 14293 tablet_service.cc:1514] Processing DeleteTablet for tablet 156a246743c24cd8a36844957700881a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:50978
I20260811 09:01:32.475903 14448 tablet_service.cc:1514] Processing DeleteTablet for tablet 156a246743c24cd8a36844957700881a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59006
I20260811 09:01:32.477998 14722 tablet_replica.cc:330] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: stopping tablet replica
I20260811 09:01:32.476879 14600 tablet_service.cc:1514] Processing DeleteTablet for tablet 156a246743c24cd8a36844957700881a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:41118
I20260811 09:01:32.478511 14722 raft_consensus.cc:2229] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:32.479066 14722 raft_consensus.cc:2258] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:32.481462 14722 ts_tablet_manager.cc:1905] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:32.482187 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14221
I20260811 09:01:32.489284 14724 tablet_replica.cc:330] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: stopping tablet replica
I20260811 09:01:32.489476 14723 tablet_replica.cc:330] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: stopping tablet replica
I20260811 09:01:32.489789 14724 raft_consensus.cc:2229] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:32.489847 14722 ts_tablet_manager.cc:1918] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:32.490078 14722 log.cc:1196] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/156a246743c24cd8a36844957700881a
I20260811 09:01:32.490043 14723 raft_consensus.cc:2229] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:32.490516 14724 raft_consensus.cc:2258] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:32.490629 14723 raft_consensus.cc:2258] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:32.490718 14722 ts_tablet_manager.cc:1939] T 156a246743c24cd8a36844957700881a P fe1bf8a344d745b4810b1258d47e89c5: Deleting consensus metadata
I20260811 09:01:32.492900 14723 ts_tablet_manager.cc:1905] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:32.493299 14130 catalog_manager.cc:4841] TS fe1bf8a344d745b4810b1258d47e89c5 (127.9.40.65:41741): tablet 156a246743c24cd8a36844957700881a (table test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]) successfully deleted
I20260811 09:01:32.495117 14724 ts_tablet_manager.cc:1905] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:32.498569 14723 ts_tablet_manager.cc:1918] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:32.498733 14723 log.cc:1196] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/156a246743c24cd8a36844957700881a
I20260811 09:01:32.499297 14723 ts_tablet_manager.cc:1939] T 156a246743c24cd8a36844957700881a P d7791e7920584c27a57f89030b70c14f: Deleting consensus metadata
I20260811 09:01:32.501147 14131 catalog_manager.cc:4841] TS d7791e7920584c27a57f89030b70c14f (127.9.40.66:34309): tablet 156a246743c24cd8a36844957700881a (table test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]) successfully deleted
I20260811 09:01:32.503244 14724 ts_tablet_manager.cc:1918] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:32.503435 14724 log.cc:1196] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/156a246743c24cd8a36844957700881a
I20260811 09:01:32.504074 14724 ts_tablet_manager.cc:1939] T 156a246743c24cd8a36844957700881a P a6c8ee1d167645c3bf97ff655ade436f: Deleting consensus metadata
Aug 11 09:01:32 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:32.505987 14130 catalog_manager.cc:4841] TS a6c8ee1d167645c3bf97ff655ade436f (127.9.40.67:35771): tablet 156a246743c24cd8a36844957700881a (table test-table [id=9f510bdc4fc945d1a26b34bc8704c2af]) successfully deleted
I20260811 09:01:32.511264 14696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.484311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.9.40.65:41741)
0811 09:01:32.484473 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.484480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.488497 (+ 4017us) server_negotiation.cc:407] Connection header received
0811 09:01:32.491189 (+ 2692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.491196 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.491304 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.491392 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:32.496851 (+ 5459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.498024 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.500203 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.500882 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.500983 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.505532 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:32.505544 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:32.505560 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:32.505593 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:32.508458 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.509256 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.509263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.509265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.509395 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.509779 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.509783 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.509785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.510139 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:32.510460 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.510714 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.510927 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:01:32.511981 14363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:32.601230 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14221
I20260811 09:01:32.630366 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14377
Aug 11 09:01:32 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:32.645988 14692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.631225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59012 (local address 127.9.40.66:34309)
0811 09:01:32.631453 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.631459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.631484 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:01:32.631630 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.631637 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.631745 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.631857 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:32.633140 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.633902 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.635133 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.635588 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.635724 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.639971 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:32.639983 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:32.639997 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:32.640034 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:32.643089 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.643734 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.643742 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.643744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.643879 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.644275 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.644281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.644284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.644770 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:32.645092 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.645391 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.645613 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:01:32.646761 14518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:32.737473 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14377
I20260811 09:01:32.764775 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14529
Aug 11 09:01:32 dist-test-slave-cbsf krb5kdc[14097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:32.782533 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:32.765774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.9.40.67:35771)
0811 09:01:32.765962 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:32.765967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:32.765988 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:01:32.766092 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:32.766103 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:32.766174 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:32.766279 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:32.767600 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.768478 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.769798 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:32.770244 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:32.770377 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:32.775286 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:32.775299 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:32.775312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:32.775349 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:32.779553 (+ 4204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.780161 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.780170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.780174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.780370 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:32.780771 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:32.780777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:32.780781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:32.781252 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:32.781634 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:32.781817 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 09:01:32.782062 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:32.783442 14669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:32.901331 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14529
I20260811 09:01:32.924058 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14112
I20260811 09:01:32.925236 14186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:33.038201 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14112
2026-08-11T09:01:33Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4366 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T09:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:33Z Disabled control of system clock
I20260811 09:01:33.113041 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45313
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:01:33.439944 14735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:33.440219 14735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:33.440301 14735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:33.447652 14735 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:33.447778 14735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:33.447822 14735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:33.447875 14735 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:33.447921 14735 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:33.457310 14735 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45313
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:33.458644 14735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:33.460731 14735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:33.473415 14740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:33.473387 14741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:33.473299 14744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:33.474298 14735 server_base.cc:1034] running on GCE node
I20260811 09:01:33.475152 14735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:33.476549 14735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:33.477808 14735 hybrid_clock.cc:648] HybridClock initialized: now 1786438893477757 us; error 61 us; skew 500 ppm
I20260811 09:01:33.478183 14735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:33.480857 14735 webserver.cc:460] Webserver started at http://127.9.40.126:37569/ using document root <none> and password file <none>
I20260811 09:01:33.481827 14735 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:33.482043 14735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:33.482400 14735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:33.485786 14735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "ff19c39cb6974d9ab40a123f6c233c3f"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "545cdb0ccfe47df7e6c244e03e9b4e82"
server_key_iv: "5f380c4d7caa46717339adce802a4db6"
server_key_version: "encryptionkey@0"
I20260811 09:01:33.486609 14735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "ff19c39cb6974d9ab40a123f6c233c3f"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "545cdb0ccfe47df7e6c244e03e9b4e82"
server_key_iv: "5f380c4d7caa46717339adce802a4db6"
server_key_version: "encryptionkey@0"
I20260811 09:01:33.491734 14735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:01:33.495340 14753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:33.496857 14735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:01:33.497044 14735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "ff19c39cb6974d9ab40a123f6c233c3f"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "545cdb0ccfe47df7e6c244e03e9b4e82"
server_key_iv: "5f380c4d7caa46717339adce802a4db6"
server_key_version: "encryptionkey@0"
I20260811 09:01:33.497259 14735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:33.503182 14735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:33.503966 14735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:33.504283 14735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:33.523991 14735 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45313
I20260811 09:01:33.524029 14809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45313 every 8 connection(s)
I20260811 09:01:33.525418 14735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:33.529294 14810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:33.531224 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14735
I20260811 09:01:33.531452 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:33.532105 9377 external_mini_cluster.cc:1428] Setting key 7e76f126e5ce57ddcce86eca14b164a8
I20260811 09:01:33.540650 14813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:33.534420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.9.40.126:45313)
0811 09:01:33.535517 (+ 1097us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:33.535527 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:33.535565 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:01:33.537240 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:33.537262 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:33.537668 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:33.537990 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:33.539138 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:33.539144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:33.539155 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:33.539171 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:33.539235 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:33.539250 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:33.539425 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:33.539527 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:01:33.539801 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":684,"thread_start_us":414,"threads_started":1}
I20260811 09:01:33.541432 14810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: Bootstrap starting.
I20260811 09:01:33.545065 14810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:33.546167 14810 log.cc:824] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:33.549033 14810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: No bootstrap required, opened a new log
I20260811 09:01:33.556823 14810 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } }
I20260811 09:01:33.557261 14810 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:33.557366 14810 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff19c39cb6974d9ab40a123f6c233c3f, State: Initialized, Role: FOLLOWER
I20260811 09:01:33.557962 14810 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } }
I20260811 09:01:33.558212 14810 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:33.558351 14810 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:33.558538 14810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:33.560622 14810 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } }
I20260811 09:01:33.561280 14810 leader_election.cc:302] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff19c39cb6974d9ab40a123f6c233c3f; no voters:
I20260811 09:01:33.561853 14810 leader_election.cc:288] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:33.561961 14815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:33.563413 14815 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [term 1 LEADER]: Becoming Leader. State: Replica: ff19c39cb6974d9ab40a123f6c233c3f, State: Running, Role: LEADER
I20260811 09:01:33.563925 14815 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } }
I20260811 09:01:33.564263 14810 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:33.566759 14817 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff19c39cb6974d9ab40a123f6c233c3f. Latest consensus state: current_term: 1 leader_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } } }
I20260811 09:01:33.566738 14816 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff19c39cb6974d9ab40a123f6c233c3f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45313 } } }
I20260811 09:01:33.567241 14816 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:33.567241 14817 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:33.569551 14823 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:33.575407 14823 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:33.583946 14823 catalog_manager.cc:1347] Generated new cluster ID: 745df67f489f486db8e77f89015b47f1
I20260811 09:01:33.584115 14823 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:33.598713 14823 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:33.599169 14823 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:33.609485 14823 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: Generated new TSK 0
I20260811 09:01:33.610239 14823 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:01:33.625619 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--builtin_ntp_servers=127.9.40.84:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:01:33.954149 14837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:33.954510 14837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:33.954705 14837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:33.962483 14837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:33.962616 14837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:33.962796 14837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:33.972307 14837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:33.973687 14837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:33.975556 14837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:33.989228 14843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:33.989182 14844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:33.989275 14846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:33.990691 14837 server_base.cc:1034] running on GCE node
I20260811 09:01:33.991423 14837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:33.992431 14837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:33.993670 14837 hybrid_clock.cc:648] HybridClock initialized: now 1786438893993583 us; error 79 us; skew 500 ppm
I20260811 09:01:33.994020 14837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:33.996337 14837 webserver.cc:460] Webserver started at http://127.9.40.65:33115/ using document root <none> and password file <none>
I20260811 09:01:33.997094 14837 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:33.997257 14837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:33.997589 14837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:34.000443 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "ffdba8d9e19d4d5392aa4376b4f4c617"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "1bee0d1491b960d97068ec7ef5ca8c8a"
server_key_iv: "9caf5ec41c8835ebe0843fa1601ec532"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.001469 14837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "ffdba8d9e19d4d5392aa4376b4f4c617"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "1bee0d1491b960d97068ec7ef5ca8c8a"
server_key_iv: "9caf5ec41c8835ebe0843fa1601ec532"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.006737 14837 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:01:34.010033 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.011314 14837 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:01:34.011487 14837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "ffdba8d9e19d4d5392aa4376b4f4c617"
format_stamp: "Formatted at 2026-08-11 09:01:33 on dist-test-slave-cbsf"
server_key: "1bee0d1491b960d97068ec7ef5ca8c8a"
server_key_iv: "9caf5ec41c8835ebe0843fa1601ec532"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.011739 14837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:34.017409 14837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:34.018186 14837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:34.018466 14837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:34.019366 14837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:34.020737 14837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:34.020855 14837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.020990 14837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:34.021076 14837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.054502 14837 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45121
I20260811 09:01:34.054546 14983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45121 every 8 connection(s)
I20260811 09:01:34.055864 14837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:34.058526 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14837
I20260811 09:01:34.058805 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:34.059803 9377 external_mini_cluster.cc:1428] Setting key 31c4273ebb934af35a42c654dfe0a6a0
I20260811 09:01:34.063166 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--builtin_ntp_servers=127.9.40.84:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 09:01:34.065562 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.058517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:38951 (local address 127.9.40.126:45313)
0811 09:01:34.060017 (+ 1500us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:34.060027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:34.060959 (+ 932us) server_negotiation.cc:407] Connection header received
0811 09:01:34.063086 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:34.063091 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:34.063142 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:34.063228 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:34.064811 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:34.064816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:34.064819 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:34.064835 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:34.064881 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:34.064893 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:34.064992 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:34.065236 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 09:01:34.065285 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":1168,"threads_started":1}
I20260811 09:01:34.066811 14987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.059090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45313 (local address 127.9.40.65:38951)
0811 09:01:34.060750 (+ 1660us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:34.060813 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:34.062773 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:34.063517 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:34.063530 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:34.063644 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:34.064367 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:34.064373 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:34.064427 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:34.064562 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:34.064996 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:34.065008 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:34.065018 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:01:34.065325 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 09:01:34.065584 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1100,"thread_start_us":858,"threads_started":1}
I20260811 09:01:34.068384 14984 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45313
I20260811 09:01:34.068775 14984 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:34.069475 14984 heartbeater.cc:511] Master 127.9.40.126:45313 requested a full tablet report, sending...
I20260811 09:01:34.071557 14772 ts_manager.cc:194] Registered new tserver with Master: ffdba8d9e19d4d5392aa4376b4f4c617 (127.9.40.65:45121)
W20260811 09:01:34.394982 14988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:34.395310 14988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:34.395480 14988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:34.402917 14988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:34.403066 14988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:34.403244 14988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:34.412117 14988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:34.413403 14988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:34.415266 14988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:34.428500 14993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:34.428558 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:34.428750 14996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:34.429281 14988 server_base.cc:1034] running on GCE node
I20260811 09:01:34.429827 14988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:34.430676 14988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:34.431908 14988 hybrid_clock.cc:648] HybridClock initialized: now 1786438894431847 us; error 49 us; skew 500 ppm
I20260811 09:01:34.432250 14988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:34.434626 14988 webserver.cc:460] Webserver started at http://127.9.40.66:35577/ using document root <none> and password file <none>
I20260811 09:01:34.435339 14988 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:34.435427 14988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:34.435773 14988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:34.438750 14988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "01b52853ca83416c858ac80dc8bda8e2"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "7a27a3123ab2e273268dc4f29b722b93"
server_key_iv: "abc103332af252d719e4fee368757252"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.439534 14988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "01b52853ca83416c858ac80dc8bda8e2"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "7a27a3123ab2e273268dc4f29b722b93"
server_key_iv: "abc103332af252d719e4fee368757252"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.444326 14988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 09:01:34.447599 15003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.448956 14988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:01:34.449127 14988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "01b52853ca83416c858ac80dc8bda8e2"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "7a27a3123ab2e273268dc4f29b722b93"
server_key_iv: "abc103332af252d719e4fee368757252"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.449361 14988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:34.454946 14988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:34.455703 14988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:34.456005 14988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:34.456961 14988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:34.458314 14988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:34.458458 14988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.458626 14988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:34.458714 14988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.493304 14988 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:34227
I20260811 09:01:34.493359 15144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:34227 every 8 connection(s)
I20260811 09:01:34.494642 14988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:34.498152 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 14988
I20260811 09:01:34.498353 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:34.498823 9377 external_mini_cluster.cc:1428] Setting key 500d89381098c8590ca7eed8b15801b9
I20260811 09:01:34.501902 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--builtin_ntp_servers=127.9.40.84:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 09:01:34.503422 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.497292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42165 (local address 127.9.40.126:45313)
0811 09:01:34.497497 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:34.497502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:34.498955 (+ 1453us) server_negotiation.cc:407] Connection header received
0811 09:01:34.500913 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:34.500927 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:34.501009 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:34.501120 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:34.502571 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:34.502580 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:34.502588 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:34.502613 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:34.502660 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:34.502679 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:34.502811 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:34.502984 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:01:34.503054 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:01:34.504179 15148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.497707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45313 (local address 127.9.40.66:42165)
0811 09:01:34.498795 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:34.498838 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:34.500605 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:34.501298 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:34.501305 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:34.501394 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:34.502274 (+ 880us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:34.502276 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:34.502323 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:34.502404 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:34.502784 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:34.502792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:34.502797 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:01:34.503005 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 09:01:34.503200 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":373,"threads_started":1}
I20260811 09:01:34.505580 15145 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45313
I20260811 09:01:34.505875 15145 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:34.506419 15145 heartbeater.cc:511] Master 127.9.40.126:45313 requested a full tablet report, sending...
I20260811 09:01:34.507882 14772 ts_manager.cc:194] Registered new tserver with Master: 01b52853ca83416c858ac80dc8bda8e2 (127.9.40.66:34227)
W20260811 09:01:34.847533 15149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:34.847788 15149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:34.847872 15149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:34.855636 15149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:34.855737 15149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:34.855842 15149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:34.864614 15149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:34.865891 15149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:34.867683 15149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:34.881139 15154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:34.881088 15157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:34.881174 15155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:34.881995 15149 server_base.cc:1034] running on GCE node
I20260811 09:01:34.882679 15149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:34.883602 15149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:34.884795 15149 hybrid_clock.cc:648] HybridClock initialized: now 1786438894884746 us; error 58 us; skew 500 ppm
I20260811 09:01:34.885130 15149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:34.887281 15149 webserver.cc:460] Webserver started at http://127.9.40.67:34905/ using document root <none> and password file <none>
I20260811 09:01:34.887943 15149 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:34.888072 15149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:34.888382 15149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:34.891173 15149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "9824152a2331476aa01381e0d9327688"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "9d3f38f3da9681a01622d3b299221f24"
server_key_iv: "701ebffd9f7d64b2be762401b612ec52"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.891939 15149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "9824152a2331476aa01381e0d9327688"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "9d3f38f3da9681a01622d3b299221f24"
server_key_iv: "701ebffd9f7d64b2be762401b612ec52"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.896876 15149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:01:34.900352 15163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.901804 15149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:34.901988 15149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "9824152a2331476aa01381e0d9327688"
format_stamp: "Formatted at 2026-08-11 09:01:34 on dist-test-slave-cbsf"
server_key: "9d3f38f3da9681a01622d3b299221f24"
server_key_iv: "701ebffd9f7d64b2be762401b612ec52"
server_key_version: "encryptionkey@0"
I20260811 09:01:34.902230 15149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:34.907850 15149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:34.908784 15149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:34.909062 15149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:34.909950 15149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:34.911301 15149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:34.911423 15149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.911556 15149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:34.911641 15149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:34.946316 15149 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:33099
I20260811 09:01:34.946403 15293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:33099 every 8 connection(s)
I20260811 09:01:34.947777 15149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:34.955900 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.950278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57549 (local address 127.9.40.126:45313)
0811 09:01:34.950463 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:34.950467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:34.951742 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 09:01:34.953647 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:34.953653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:34.953733 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:34.953866 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:34.955117 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:34.955123 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:34.955128 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:34.955148 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:34.955190 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:34.955204 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:34.955355 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:34.955540 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 09:01:34.955612 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:01:34.956470 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15149
I20260811 09:01:34.956663 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:34.956843 15297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.950674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45313 (local address 127.9.40.67:57549)
0811 09:01:34.951581 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:34.951623 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:34.953381 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:34.954096 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:34.954122 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:34.954205 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:34.954815 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:34.954817 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:34.954853 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:34.954931 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:34.955318 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:34.955330 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:34.955339 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:01:34.955619 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 09:01:34.955848 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":270,"threads_started":1}
I20260811 09:01:34.957237 9377 external_mini_cluster.cc:1428] Setting key b71512d9f0bcab8a3c08f998b308350e
I20260811 09:01:34.958431 15294 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45313
I20260811 09:01:34.958781 15294 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:34.959451 15294 heartbeater.cc:511] Master 127.9.40.126:45313 requested a full tablet report, sending...
I20260811 09:01:34.960947 14772 ts_manager.cc:194] Registered new tserver with Master: 9824152a2331476aa01381e0d9327688 (127.9.40.67:33099)
I20260811 09:01:34.972769 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:01:34.979578 14986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:34.977273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35694 (local address 127.9.40.126:45313)
0811 09:01:34.977436 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:34.977440 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:34.977928 (+ 488us) server_negotiation.cc:407] Connection header received
0811 09:01:34.978064 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:34.978068 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:34.978121 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:34.978203 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:34.979147 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:34.979151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:34.979154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:34.979167 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:34.979194 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:34.979205 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:34.979294 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:34.979330 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 09:01:34.979372 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:01:34.983508 14772 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:34.990185 14772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:35.029064 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.011456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:34227 (local address 127.0.0.1:44928)
0811 09:01:35.023095 (+ 11639us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.023183 (+ 88us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:35.023339 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:35.025514 (+ 2175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:35.025525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:35.025641 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:35.026582 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:35.026588 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:35.026628 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:35.026732 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:35.028333 (+ 1601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:35.028343 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:35.028352 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:01:35.028528 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 09:01:35.028616 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11303,"thread_start_us":12017,"threads_started":1}
I20260811 09:01:35.029976 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.011662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.9.40.66:34227)
0811 09:01:35.024789 (+ 13127us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.024800 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.024834 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:01:35.024926 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.024938 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.025251 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.025469 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.026908 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.026920 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.026938 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.026964 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.027033 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.027057 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.027222 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.028570 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.028721 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12935,"thread_start_us":14942,"threads_started":1}
I20260811 09:01:35.033133 15309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.011718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33099 (local address 127.0.0.1:52030)
0811 09:01:35.024153 (+ 12435us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.024176 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:35.024278 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:35.029511 (+ 5233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:35.029519 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:35.029553 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:35.030314 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:35.030318 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:35.030337 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:35.030382 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:35.030974 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:35.030982 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:35.030990 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:01:35.031160 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 09:01:35.032829 (+ 1669us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12286,"thread_start_us":12129,"threads_started":1}
I20260811 09:01:35.033712 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.012409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.9.40.67:33099)
0811 09:01:35.028291 (+ 15882us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.028301 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.028332 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:01:35.028396 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.028402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.028614 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.028937 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.030579 (+ 1642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.030587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.030599 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.030617 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.030676 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.030697 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.030906 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.033359 (+ 2453us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.033445 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15693,"thread_start_us":16293,"threads_started":1}
I20260811 09:01:35.034803 15067 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b (DEFAULT_TABLE table=test-table [id=8b15c579c1e847cf96ce3004cb49998f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:35.036590 15067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba14e94188e4f60bfa4d2e36a35f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:35.036983 15221 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b (DEFAULT_TABLE table=test-table [id=8b15c579c1e847cf96ce3004cb49998f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:35.038311 15221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba14e94188e4f60bfa4d2e36a35f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:35.039323 15307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.030611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51590)
0811 09:01:35.030778 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.031266 (+ 488us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:35.031340 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:35.037824 (+ 6484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:35.037829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:35.037866 (+ 37us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:35.038513 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:35.038517 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:35.038533 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:35.038585 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:35.038946 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:35.038951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:35.038955 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:01:35.039048 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:01:35.039096 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:01:35.039446 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.024663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.9.40.65:45121)
0811 09:01:35.037337 (+ 12674us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.037347 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.037376 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:35.037444 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.037451 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.037637 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.037814 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.038714 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.038725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.038739 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.038756 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.038809 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.038827 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.038990 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.039055 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.039147 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12479,"threads_started":1}
I20260811 09:01:35.042603 14909 tablet_service.cc:1467] Processing CreateTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b (DEFAULT_TABLE table=test-table [id=8b15c579c1e847cf96ce3004cb49998f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:35.043767 14909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ba14e94188e4f60bfa4d2e36a35f21b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:35.049690 15312 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Bootstrap starting.
I20260811 09:01:35.053417 15312 tablet_bootstrap.cc:654] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:35.054845 15312 log.cc:824] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:35.060552 15313 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Bootstrap starting.
I20260811 09:01:35.065482 15315 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Bootstrap starting.
I20260811 09:01:35.067246 15313 tablet_bootstrap.cc:654] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:35.067385 15312 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: No bootstrap required, opened a new log
I20260811 09:01:35.067687 15312 ts_tablet_manager.cc:1397] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.005s
I20260811 09:01:35.068372 15313 log.cc:824] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:35.073269 15315 tablet_bootstrap.cc:654] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:35.074239 14984 heartbeater.cc:503] Master 127.9.40.126:45313 was elected leader, sending a full tablet report...
I20260811 09:01:35.074972 15315 log.cc:824] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:35.076612 15312 raft_consensus.cc:348] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.077230 15312 raft_consensus.cc:374] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:35.077394 15312 raft_consensus.cc:729] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9824152a2331476aa01381e0d9327688, State: Initialized, Role: FOLLOWER
I20260811 09:01:35.078192 15312 consensus_queue.cc:260] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.078477 15313 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: No bootstrap required, opened a new log
I20260811 09:01:35.078794 15313 ts_tablet_manager.cc:1397] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20260811 09:01:35.079983 15294 heartbeater.cc:503] Master 127.9.40.126:45313 was elected leader, sending a full tablet report...
I20260811 09:01:35.080997 15312 ts_tablet_manager.cc:1428] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 09:01:35.081593 15315 tablet_bootstrap.cc:492] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: No bootstrap required, opened a new log
I20260811 09:01:35.081856 15315 ts_tablet_manager.cc:1397] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.008s
I20260811 09:01:35.087674 15313 raft_consensus.cc:348] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.088039 15313 raft_consensus.cc:374] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:35.088155 15313 raft_consensus.cc:729] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01b52853ca83416c858ac80dc8bda8e2, State: Initialized, Role: FOLLOWER
I20260811 09:01:35.088665 15313 consensus_queue.cc:260] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.089777 15145 heartbeater.cc:503] Master 127.9.40.126:45313 was elected leader, sending a full tablet report...
I20260811 09:01:35.089941 15315 raft_consensus.cc:348] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.090313 15315 raft_consensus.cc:374] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:35.090340 15313 ts_tablet_manager.cc:1428] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 09:01:35.090425 15315 raft_consensus.cc:729] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffdba8d9e19d4d5392aa4376b4f4c617, State: Initialized, Role: FOLLOWER
I20260811 09:01:35.090987 15315 consensus_queue.cc:260] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.092644 15315 ts_tablet_manager.cc:1428] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
W20260811 09:01:35.199519 15295 tablet.cc:2369] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:35.253301 15147 tablet.cc:2369] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:35.308400 14985 tablet.cc:2369] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:35.357606 15320 raft_consensus.cc:482] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:35.357980 15320 raft_consensus.cc:504] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.359735 15320 leader_election.cc:288] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01b52853ca83416c858ac80dc8bda8e2 (127.9.40.66:34227), 9824152a2331476aa01381e0d9327688 (127.9.40.67:33099)
I20260811 09:01:35.363327 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.360323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35133 (local address 127.9.40.67:33099)
0811 09:01:35.360517 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.360524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.361335 (+ 811us) server_negotiation.cc:407] Connection header received
0811 09:01:35.361548 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.361553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.361613 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.361711 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.362704 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.362710 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.362715 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.362733 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.362774 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.362786 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.362920 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.362998 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.363065 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:01:35.363481 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.360256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:52345 (local address 127.9.40.66:34227)
0811 09:01:35.360470 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.360477 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.361712 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 09:01:35.361846 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.361852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.361907 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.362011 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.362852 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.362857 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.362862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.362879 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.362916 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.362928 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.363046 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.363140 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.363202 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:35.363554 15322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.360189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33099 (local address 127.9.40.65:35133)
0811 09:01:35.361214 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.361235 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:35.361391 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:35.361741 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:35.361747 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:35.361770 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:35.362460 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:35.362463 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:35.362484 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:35.362536 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:35.362881 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:35.362886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:35.362890 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:01:35.363040 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:01:35.363091 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":903,"thread_start_us":653,"threads_started":1}
I20260811 09:01:35.363739 15323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.360656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:34227 (local address 127.9.40.65:52345)
0811 09:01:35.361607 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.361620 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:35.361741 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:35.362041 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:35.362048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:35.362078 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:35.362665 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:01:35.362667 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:35.362679 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:01:35.362710 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:35.363025 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:35.363029 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:35.363033 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:01:35.363188 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:01:35.363239 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":527,"threads_started":1}
I20260811 09:01:35.364661 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba14e94188e4f60bfa4d2e36a35f21b" candidate_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9824152a2331476aa01381e0d9327688" is_pre_election: true
I20260811 09:01:35.364761 15089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba14e94188e4f60bfa4d2e36a35f21b" candidate_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01b52853ca83416c858ac80dc8bda8e2" is_pre_election: true
I20260811 09:01:35.365302 15089 raft_consensus.cc:2454] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffdba8d9e19d4d5392aa4376b4f4c617 in term 0.
I20260811 09:01:35.365320 15244 raft_consensus.cc:2454] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffdba8d9e19d4d5392aa4376b4f4c617 in term 0.
I20260811 09:01:35.365988 14855 leader_election.cc:302] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01b52853ca83416c858ac80dc8bda8e2, ffdba8d9e19d4d5392aa4376b4f4c617; no voters:
I20260811 09:01:35.366379 15320 raft_consensus.cc:2781] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:35.366523 15320 raft_consensus.cc:482] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:35.366624 15320 raft_consensus.cc:3037] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:35.368960 15320 raft_consensus.cc:504] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.369694 15320 leader_election.cc:288] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [CANDIDATE]: Term 1 election: Requested vote from peers 01b52853ca83416c858ac80dc8bda8e2 (127.9.40.66:34227), 9824152a2331476aa01381e0d9327688 (127.9.40.67:33099)
I20260811 09:01:35.370095 15089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba14e94188e4f60bfa4d2e36a35f21b" candidate_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01b52853ca83416c858ac80dc8bda8e2"
I20260811 09:01:35.370158 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ba14e94188e4f60bfa4d2e36a35f21b" candidate_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9824152a2331476aa01381e0d9327688"
I20260811 09:01:35.370326 15089 raft_consensus.cc:3037] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:35.370381 15244 raft_consensus.cc:3037] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:35.372552 15244 raft_consensus.cc:2454] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffdba8d9e19d4d5392aa4376b4f4c617 in term 1.
I20260811 09:01:35.372550 15089 raft_consensus.cc:2454] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffdba8d9e19d4d5392aa4376b4f4c617 in term 1.
I20260811 09:01:35.373027 14857 leader_election.cc:302] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9824152a2331476aa01381e0d9327688, ffdba8d9e19d4d5392aa4376b4f4c617; no voters:
I20260811 09:01:35.373359 15320 raft_consensus.cc:2781] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:35.373871 15320 raft_consensus.cc:686] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [term 1 LEADER]: Becoming Leader. State: Replica: ffdba8d9e19d4d5392aa4376b4f4c617, State: Running, Role: LEADER
I20260811 09:01:35.374389 15320 consensus_queue.cc:237] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } }
I20260811 09:01:35.379369 14771 catalog_manager.cc:5495] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 reported cstate change: term changed from 0 to 1, leader changed from <none> to ffdba8d9e19d4d5392aa4376b4f4c617 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ffdba8d9e19d4d5392aa4376b4f4c617" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45121 } health_report { overall_health: HEALTHY } } }
I20260811 09:01:35.408269 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.406028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.9.40.65:45121)
0811 09:01:35.406220 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.406227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.406315 (+ 88us) server_negotiation.cc:407] Connection header received
0811 09:01:35.406579 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.406584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.406662 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.406779 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.407638 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.407646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.407649 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.407670 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.407701 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.407712 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.407835 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.407937 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.407996 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:01:35.416381 15244 raft_consensus.cc:1261] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 1 FOLLOWER]: Refusing update from remote peer ffdba8d9e19d4d5392aa4376b4f4c617: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:35.416414 15089 raft_consensus.cc:1261] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 1 FOLLOWER]: Refusing update from remote peer ffdba8d9e19d4d5392aa4376b4f4c617: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:35.417541 15325 consensus_queue.cc:1035] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01b52853ca83416c858ac80dc8bda8e2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:35.417906 15320 consensus_queue.cc:1035] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9824152a2331476aa01381e0d9327688" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:35.445266 14772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:35.445577 14772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:35.448750 14772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff19c39cb6974d9ab40a123f6c233c3f: Sending DeleteTablet for 3 replicas of tablet 6ba14e94188e4f60bfa4d2e36a35f21b
I20260811 09:01:35.449756 15067 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:35 UTC) from {username='slave'} at 127.0.0.1:44928
I20260811 09:01:35.450011 14909 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:35 UTC) from {username='slave'} at 127.0.0.1:51590
I20260811 09:01:35.449995 15221 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ba14e94188e4f60bfa4d2e36a35f21b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:35 UTC) from {username='slave'} at 127.0.0.1:52030
I20260811 09:01:35.453058 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14837
I20260811 09:01:35.472615 15311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.455861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51596 (local address 127.9.40.65:45121)
0811 09:01:35.456077 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.456086 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.469266 (+ 13180us) server_negotiation.cc:407] Connection header received
0811 09:01:35.469599 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.469608 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.469713 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.469807 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.471306 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.471315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.471323 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.471348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.471397 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.471437 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.471602 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.472119 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.472194 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 09:01:35.473534 15339 tablet_replica.cc:330] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: stopping tablet replica
I20260811 09:01:35.473958 14979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:35.474382 15339 raft_consensus.cc:2229] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:35.475124 15339 raft_consensus.cc:2258] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:35.476449 15342 tablet_replica.cc:330] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: stopping tablet replica
I20260811 09:01:35.478381 15339 ts_tablet_manager.cc:1905] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:35.480203 15342 raft_consensus.cc:2229] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:35.480712 15342 raft_consensus.cc:2258] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:35.482908 15342 ts_tablet_manager.cc:1905] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:35.486781 15339 ts_tablet_manager.cc:1918] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:35.486974 15339 log.cc:1196] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/6ba14e94188e4f60bfa4d2e36a35f21b
I20260811 09:01:35.487514 15339 ts_tablet_manager.cc:1939] T 6ba14e94188e4f60bfa4d2e36a35f21b P 01b52853ca83416c858ac80dc8bda8e2: Deleting consensus metadata
I20260811 09:01:35.489357 14754 catalog_manager.cc:4841] TS 01b52853ca83416c858ac80dc8bda8e2 (127.9.40.66:34227): tablet 6ba14e94188e4f60bfa4d2e36a35f21b (table test-table [id=8b15c579c1e847cf96ce3004cb49998f]) successfully deleted
I20260811 09:01:35.490906 15342 ts_tablet_manager.cc:1918] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:35.491092 15342 log.cc:1196] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/6ba14e94188e4f60bfa4d2e36a35f21b
I20260811 09:01:35.491528 15342 ts_tablet_manager.cc:1939] T 6ba14e94188e4f60bfa4d2e36a35f21b P 9824152a2331476aa01381e0d9327688: Deleting consensus metadata
I20260811 09:01:35.493053 14756 catalog_manager.cc:4841] TS 9824152a2331476aa01381e0d9327688 (127.9.40.67:33099): tablet 6ba14e94188e4f60bfa4d2e36a35f21b (table test-table [id=8b15c579c1e847cf96ce3004cb49998f]) successfully deleted
I20260811 09:01:35.574399 15340 tablet_replica.cc:330] T 6ba14e94188e4f60bfa4d2e36a35f21b P ffdba8d9e19d4d5392aa4376b4f4c617: stopping tablet replica
I20260811 09:01:35.574440 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14837
I20260811 09:01:35.597966 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14988
W20260811 09:01:35.597990 14756 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:45121 (error 108)
I20260811 09:01:35.600234 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.599500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51600)
0811 09:01:35.599926 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.600084 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":262,"threads_started":1}
W20260811 09:01:35.600628 14756 catalog_manager.cc:4584] TS ffdba8d9e19d4d5392aa4376b4f4c617 (127.9.40.65:45121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ba14e94188e4f60bfa4d2e36a35f21b: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
I20260811 09:01:35.601294 15308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.598828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44936 (local address 127.9.40.66:34227)
0811 09:01:35.599048 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.599054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.599213 (+ 159us) server_negotiation.cc:407] Connection header received
0811 09:01:35.599404 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.599408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.599500 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.599581 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.600612 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.600616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.600619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.600633 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.600664 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.600675 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.600834 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.601012 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.601097 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:01:35.601970 15139 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:35.654619 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.653954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51606)
0811 09:01:35.654251 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.654403 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 09:01:35.655823 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.655406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51608)
0811 09:01:35.655577 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.655664 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:01:35.690580 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14988
I20260811 09:01:35.711935 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15149
I20260811 09:01:35.714632 15310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.712755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.9.40.67:33099)
0811 09:01:35.712907 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:35.712912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:35.712929 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:01:35.713004 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:35.713008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:35.713101 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:35.713183 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:35.714068 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:35.714072 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:35.714075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:35.714088 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:35.714116 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:35.714128 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:35.714250 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:35.714331 (+ 81us) server_negotiation.cc:299] Negotiation successful
0811 09:01:35.714393 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 09:01:35.715251 15289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:35.716426 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.715960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51614)
0811 09:01:35.716162 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.716259 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:01:35.717655 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.717264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51620)
0811 09:01:35.717436 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.717513 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:01:35.798516 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.797980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51630)
0811 09:01:35.798225 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.798323 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 09:01:35.799687 15347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:35.799271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45121 (local address 127.0.0.1:51642)
0811 09:01:35.799489 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:35.799558 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 09:01:35.803499 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15149
I20260811 09:01:35.822280 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 14735
I20260811 09:01:35.823565 14805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:35.929333 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 14735
2026-08-11T09:01:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2891 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 09:01:35.976334 9377 mini_oidc.cc:152] Starting JWKS server
I20260811 09:01:35.979465 9377 webserver.cc:460] Webserver started at https://127.0.0.1:42893/ using document root <none> and password file <none>
I20260811 09:01:35.979799 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:01:35.980494 9377 webserver.cc:460] Webserver started at http://127.0.0.1:37243/ using document root <none> and password file <none>
2026-08-11T09:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:35Z Disabled control of system clock
I20260811 09:01:36.012266 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34905
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:41591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:34905
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42893/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/testchainca.cert with env {}
W20260811 09:01:36.339097 15357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:36.339480 15357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:36.339653 15357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:36.346732 15357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:36.346913 15357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:36.347018 15357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:36.347126 15357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:36.347216 15357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:01:36.347314 15357 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:01:36.347393 15357 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42893/jwks.json
I20260811 09:01:36.357055 15357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:34905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42893/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:36.358361 15357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:36.359764 15357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:36.371874 15364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:36.371964 15367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:36.371850 15365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:36.372670 15357 server_base.cc:1034] running on GCE node
I20260811 09:01:36.373517 15357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:36.374890 15357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:36.376129 15357 hybrid_clock.cc:648] HybridClock initialized: now 1786438896376095 us; error 46 us; skew 500 ppm
I20260811 09:01:36.376483 15357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:36.378790 15357 webserver.cc:460] Webserver started at http://127.9.40.126:41355/ using document root <none> and password file <none>
I20260811 09:01:36.379465 15357 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:36.379616 15357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:36.379936 15357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:36.382886 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "b09ec0c0dd72404f8534f5644b5765f7"
format_stamp: "Formatted at 2026-08-11 09:01:36 on dist-test-slave-cbsf"
server_key: "e85bf46fc08e8f02330d2730e30bda0d"
server_key_iv: "dc02c588f8b446ce18e2e7637e760f32"
server_key_version: "encryptionkey@0"
I20260811 09:01:36.383812 15357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "b09ec0c0dd72404f8534f5644b5765f7"
format_stamp: "Formatted at 2026-08-11 09:01:36 on dist-test-slave-cbsf"
server_key: "e85bf46fc08e8f02330d2730e30bda0d"
server_key_iv: "dc02c588f8b446ce18e2e7637e760f32"
server_key_version: "encryptionkey@0"
I20260811 09:01:36.388758 15357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 09:01:36.392129 15375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:36.393497 15357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:01:36.393695 15357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "b09ec0c0dd72404f8534f5644b5765f7"
format_stamp: "Formatted at 2026-08-11 09:01:36 on dist-test-slave-cbsf"
server_key: "e85bf46fc08e8f02330d2730e30bda0d"
server_key_iv: "dc02c588f8b446ce18e2e7637e760f32"
server_key_version: "encryptionkey@0"
I20260811 09:01:36.393909 15357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:36.413450 15357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:36.414295 15357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:36.414603 15357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:36.433676 15357 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:34905
I20260811 09:01:36.433698 15437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:34905 every 8 connection(s)
I20260811 09:01:36.435025 15357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:36.438889 15438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:36.442158 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15357
I20260811 09:01:36.442370 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:36.443233 9377 external_mini_cluster.cc:1428] Setting key c271de45eaa4a52819270d1ac921f027
I20260811 09:01:36.451455 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7: Bootstrap starting.
I20260811 09:01:36.455037 15438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:36.456169 15438 log.cc:824] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:36.456357 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:36.445185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.9.40.126:34905)
0811 09:01:36.446115 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:36.446126 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:36.446164 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:01:36.447717 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:36.447738 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:36.448159 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:36.448479 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:36.449162 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.450362 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.452116 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.452783 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.452916 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:36.454168 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:36.454175 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:36.454190 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:36.454693 (+ 503us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:36.454780 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:36.454800 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:36.454883 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:36.455196 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 09:01:36.455462 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":373,"threads_started":1}
I20260811 09:01:36.460026 15438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7: No bootstrap required, opened a new log
I20260811 09:01:36.460093 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:01:36.467670 15438 raft_consensus.cc:348] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } }
I20260811 09:01:36.468127 15438 raft_consensus.cc:374] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:36.468257 15438 raft_consensus.cc:729] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b09ec0c0dd72404f8534f5644b5765f7, State: Initialized, Role: FOLLOWER
I20260811 09:01:36.468896 15438 consensus_queue.cc:260] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } }
I20260811 09:01:36.469136 15438 raft_consensus.cc:388] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:36.469285 15438 raft_consensus.cc:482] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:36.469465 15438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:36.471519 15438 raft_consensus.cc:504] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } }
I20260811 09:01:36.472026 15438 leader_election.cc:302] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b09ec0c0dd72404f8534f5644b5765f7; no voters:
I20260811 09:01:36.472553 15438 leader_election.cc:288] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:36.472728 15443 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:36.473412 15443 raft_consensus.cc:686] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [term 1 LEADER]: Becoming Leader. State: Replica: b09ec0c0dd72404f8534f5644b5765f7, State: Running, Role: LEADER
I20260811 09:01:36.473874 15443 consensus_queue.cc:237] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } }
I20260811 09:01:36.474572 15438 sys_catalog.cc:564] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:36.476749 15444 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b09ec0c0dd72404f8534f5644b5765f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } } }
I20260811 09:01:36.476595 15445 sys_catalog.cc:455] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b09ec0c0dd72404f8534f5644b5765f7. Latest consensus state: current_term: 1 leader_uuid: "b09ec0c0dd72404f8534f5644b5765f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b09ec0c0dd72404f8534f5644b5765f7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34905 } } }
I20260811 09:01:36.477262 15445 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:36.477272 15444 sys_catalog.cc:458] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:36.479094 15451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:36.484344 15451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:36.494131 15451 catalog_manager.cc:1347] Generated new cluster ID: 4286a481e84840e38d6130107a1d4e37
I20260811 09:01:36.494303 15451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:36.502799 15451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:36.504547 15451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:36.511889 15451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b09ec0c0dd72404f8534f5644b5765f7: Generated new TSK 0
I20260811 09:01:36.512669 15451 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:01:36.564286 15474 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 09:01:36.564502 15474 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:01:36.565775 15441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:36.535663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.9.40.126:34905)
0811 09:01:36.535872 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:36.535879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:36.536238 (+ 359us) server_negotiation.cc:407] Connection header received
0811 09:01:36.536617 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:36.536621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:36.536686 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:36.536868 (+ 182us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:01:36.537586 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.538495 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.540582 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.541166 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.541281 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:36.541947 (+ 666us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:01:36.564973 (+ 23026us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:01:36.565452 (+ 479us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48766: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 09:01:36.565980 15441 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48766: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 09:01:36.567726 15473 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:36.535394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:34905 (local address 127.0.0.1:48766)
0811 09:01:36.536120 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:36.536140 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:36.536326 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:36.536923 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:36.536929 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:36.536945 (+ 16us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:01:36.537376 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:36.537390 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.538656 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.538662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:36.540413 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:36.540429 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:36.541364 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:36.541371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:36.541588 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:36.541670 (+ 82us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:01:36.565315 (+ 23645us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:01:36.565740 (+ 425us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:34905: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":353,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.40.126:34905: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42893/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 09:01:36.570009 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15357
I20260811 09:01:36.571192 15432 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:36.663663 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15357
2026-08-11T09:01:36Z chronyd exiting
I20260811 09:01:36.718602 9377 test_util.cc:175] -----------------------------------------------
I20260811 09:01:36.718722 9377 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786438852478101-9377-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (747 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 09:01:36.722595 9377 mini_oidc.cc:152] Starting JWKS server
I20260811 09:01:36.724436 9377 webserver.cc:460] Webserver started at https://127.0.0.1:45679/ using document root <none> and password file <none>
I20260811 09:01:36.724655 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:01:36.725325 9377 webserver.cc:460] Webserver started at http://127.0.0.1:36357/ using document root <none> and password file <none>
2026-08-11T09:01:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:36Z Disabled control of system clock
I20260811 09:01:36.759028 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41019
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:44837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41019
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45679/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/testchainca.cert with env {}
W20260811 09:01:37.089097 15481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:37.089380 15481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:37.089468 15481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:37.096536 15481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:37.096647 15481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:37.096685 15481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:37.096828 15481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:37.096881 15481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:01:37.096940 15481 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:01:37.096995 15481 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45679/jwks.json
I20260811 09:01:37.107290 15481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:44837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45679/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:37.108628 15481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:37.109936 15481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:37.121809 15490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:37.121873 15491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:37.121840 15494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:37.122843 15481 server_base.cc:1034] running on GCE node
I20260811 09:01:37.123600 15481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:37.125002 15481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:37.126269 15481 hybrid_clock.cc:648] HybridClock initialized: now 1786438897126191 us; error 44 us; skew 500 ppm
I20260811 09:01:37.126626 15481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:37.133009 15481 webserver.cc:460] Webserver started at http://127.9.40.126:41873/ using document root <none> and password file <none>
I20260811 09:01:37.133733 15481 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:37.133872 15481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:37.134214 15481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:37.137225 15481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "79051deff225402589a31647c789c911"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "1e16d7f6cba61b41c8836d47ca468d3d"
server_key_iv: "ac3f327198a6acd74d6fcde6eae03d38"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.138041 15481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "79051deff225402589a31647c789c911"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "1e16d7f6cba61b41c8836d47ca468d3d"
server_key_iv: "ac3f327198a6acd74d6fcde6eae03d38"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.143465 15481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:01:37.147311 15501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:37.149185 15481 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 09:01:37.149386 15481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "79051deff225402589a31647c789c911"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "1e16d7f6cba61b41c8836d47ca468d3d"
server_key_iv: "ac3f327198a6acd74d6fcde6eae03d38"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.149679 15481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:37.180433 15481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:37.181366 15481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:37.181655 15481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:37.200098 15481 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41019
I20260811 09:01:37.200134 15556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41019 every 8 connection(s)
I20260811 09:01:37.201462 15481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:37.205176 15557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:37.209472 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15481
I20260811 09:01:37.209708 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:37.210538 9377 external_mini_cluster.cc:1428] Setting key 343cfddce18c316be2a9476de06ca717
I20260811 09:01:37.216918 15557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911: Bootstrap starting.
I20260811 09:01:37.219888 15557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:37.221112 15557 log.cc:824] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:37.224416 15557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911: No bootstrap required, opened a new log
I20260811 09:01:37.224818 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:37.212539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35270 (local address 127.9.40.126:41019)
0811 09:01:37.213649 (+ 1110us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:37.213663 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:37.213703 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:01:37.215584 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:37.215605 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:37.216034 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:37.216405 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:37.216971 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.218344 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.219626 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.220431 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.220626 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:37.221984 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:37.221992 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:37.222005 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:37.222671 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:37.222792 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:37.222825 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:37.222951 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:37.223176 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 09:01:37.223529 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":473,"threads_started":1}
I20260811 09:01:37.228791 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:01:37.234084 15557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } }
I20260811 09:01:37.234452 15557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:37.234606 15557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79051deff225402589a31647c789c911, State: Initialized, Role: FOLLOWER
I20260811 09:01:37.235215 15557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } }
I20260811 09:01:37.235431 15557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:37.235533 15557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:37.235657 15557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:37.238319 15557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } }
I20260811 09:01:37.238914 15557 leader_election.cc:302] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79051deff225402589a31647c789c911; no voters:
I20260811 09:01:37.239609 15557 leader_election.cc:288] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:37.239712 15563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:37.240448 15563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [term 1 LEADER]: Becoming Leader. State: Replica: 79051deff225402589a31647c789c911, State: Running, Role: LEADER
I20260811 09:01:37.241074 15563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } }
I20260811 09:01:37.241878 15557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:37.243616 15565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79051deff225402589a31647c789c911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } } }
I20260811 09:01:37.243939 15565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:37.244668 15566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79051deff225402589a31647c789c911. Latest consensus state: current_term: 1 leader_uuid: "79051deff225402589a31647c789c911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79051deff225402589a31647c789c911" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41019 } } }
I20260811 09:01:37.245033 15566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:37.246315 15570 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:37.251432 15570 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:37.261052 15570 catalog_manager.cc:1347] Generated new cluster ID: 0039a8db7ed34513973f27792c7d90ae
I20260811 09:01:37.261214 15570 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:37.296058 15570 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:37.298002 15570 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:37.305130 15570 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79051deff225402589a31647c789c911: Generated new TSK 0
I20260811 09:01:37.306308 15570 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:01:37.338116 15594 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 09:01:37.338258 15594 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:01:37.339660 15561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:37.319156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35272 (local address 127.9.40.126:41019)
0811 09:01:37.319352 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:37.319360 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:37.319834 (+ 474us) server_negotiation.cc:407] Connection header received
0811 09:01:37.320084 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:37.320092 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:37.320194 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:37.320364 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:01:37.320902 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.322021 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.323379 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.324131 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.324269 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:37.324778 (+ 509us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:01:37.338857 (+ 14079us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:01:37.339302 (+ 445us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:01:37.339807 15561 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35272: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 09:01:37.339854 15593 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:37.319116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41019 (local address 127.0.0.1:35272)
0811 09:01:37.319738 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:37.319753 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:37.319865 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:37.320386 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:37.320389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:37.320397 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:01:37.320733 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:37.320748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.322187 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.322191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:37.323236 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:37.323246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.324290 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.324294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:37.324468 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:37.324499 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:01:37.339129 (+ 14630us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:01:37.339333 (+ 204us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:41019: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45679/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":357,"threads_started":1}
I20260811 09:01:37.340895 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15481
I20260811 09:01:37.342485 15551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:37.437372 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15481
2026-08-11T09:01:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (775 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 09:01:37.498881 9377 mini_oidc.cc:152] Starting JWKS server
I20260811 09:01:37.500890 9377 webserver.cc:460] Webserver started at https://127.0.0.1:43241/ using document root <none> and password file <none>
I20260811 09:01:37.501166 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:01:37.501860 9377 webserver.cc:460] Webserver started at http://127.0.0.1:37157/ using document root <none> and password file <none>
2026-08-11T09:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:37Z Disabled control of system clock
I20260811 09:01:37.535036 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40697
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:46809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40697
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43241/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:01:37.864202 15601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:37.864569 15601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:37.864768 15601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:37.871847 15601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:37.871989 15601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:37.872054 15601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:37.872161 15601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:37.872260 15601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:01:37.872350 15601 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:01:37.872406 15601 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43241/jwks.json
I20260811 09:01:37.881490 15601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43241/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:37.882879 15601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:37.884681 15601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:37.896462 15612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:37.896523 15608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:37.896543 15610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:37.897614 15601 server_base.cc:1034] running on GCE node
I20260811 09:01:37.898391 15601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:37.899792 15601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:37.901021 15601 hybrid_clock.cc:648] HybridClock initialized: now 1786438897900966 us; error 53 us; skew 500 ppm
I20260811 09:01:37.901373 15601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:37.903745 15601 webserver.cc:460] Webserver started at http://127.9.40.126:37519/ using document root <none> and password file <none>
I20260811 09:01:37.904464 15601 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:37.904606 15601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:37.904979 15601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:37.907915 15601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "c9f650396c2d412b8ebbb0b661598399"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "c5e67b716e7a1ade90b3c6225f5a8794"
server_key_iv: "b314152905b1cf09c2890429e7bcffc4"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.908844 15601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "c9f650396c2d412b8ebbb0b661598399"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "c5e67b716e7a1ade90b3c6225f5a8794"
server_key_iv: "b314152905b1cf09c2890429e7bcffc4"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.913874 15601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:01:37.917232 15619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:37.918543 15601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:01:37.918738 15601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "c9f650396c2d412b8ebbb0b661598399"
format_stamp: "Formatted at 2026-08-11 09:01:37 on dist-test-slave-cbsf"
server_key: "c5e67b716e7a1ade90b3c6225f5a8794"
server_key_iv: "b314152905b1cf09c2890429e7bcffc4"
server_key_version: "encryptionkey@0"
I20260811 09:01:37.918977 15601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:37.949659 15601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:37.950546 15601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:37.950860 15601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:37.969898 15601 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:40697
I20260811 09:01:37.969923 15675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:40697 every 8 connection(s)
I20260811 09:01:37.971298 15601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:37.975364 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15601
I20260811 09:01:37.975414 15676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:37.975603 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:37.976557 9377 external_mini_cluster.cc:1428] Setting key efcc515b445030f4ba99ec087570adbe
I20260811 09:01:37.989214 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399: Bootstrap starting.
I20260811 09:01:37.990049 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:37.978735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46544 (local address 127.9.40.126:40697)
0811 09:01:37.979833 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:37.979843 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:37.979884 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:01:37.981465 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:37.981485 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:37.981913 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:37.982319 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:37.982884 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.984154 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.985595 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:37.986304 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:37.986530 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:37.987727 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:37.987735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:37.987749 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:37.988291 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:37.988366 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:37.988387 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:37.988523 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:37.988794 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 09:01:37.989095 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":427,"threads_started":1}
I20260811 09:01:37.992828 15676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:37.994194 15676 log.cc:824] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:37.995324 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:01:37.998044 15676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399: No bootstrap required, opened a new log
I20260811 09:01:38.010581 15676 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } }
I20260811 09:01:38.011114 15676 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:38.011233 15676 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f650396c2d412b8ebbb0b661598399, State: Initialized, Role: FOLLOWER
I20260811 09:01:38.011914 15676 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } }
I20260811 09:01:38.012233 15676 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:38.012375 15676 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:38.012578 15676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:38.016407 15676 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } }
I20260811 09:01:38.017092 15676 leader_election.cc:302] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9f650396c2d412b8ebbb0b661598399; no voters:
I20260811 09:01:38.017874 15676 leader_election.cc:288] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:38.018046 15682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:38.018908 15682 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [term 1 LEADER]: Becoming Leader. State: Replica: c9f650396c2d412b8ebbb0b661598399, State: Running, Role: LEADER
I20260811 09:01:38.019492 15682 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } }
I20260811 09:01:38.020392 15676 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:38.022522 15683 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9f650396c2d412b8ebbb0b661598399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } } }
I20260811 09:01:38.023087 15683 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:38.022917 15684 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9f650396c2d412b8ebbb0b661598399. Latest consensus state: current_term: 1 leader_uuid: "c9f650396c2d412b8ebbb0b661598399" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9f650396c2d412b8ebbb0b661598399" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40697 } } }
I20260811 09:01:38.023212 15684 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:38.023947 15687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:38.033500 15687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:38.043020 15687 catalog_manager.cc:1347] Generated new cluster ID: d0a3813d658e4d3a9dea8829eca55153
I20260811 09:01:38.043152 15687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:38.058024 15687 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:38.059293 15687 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:38.072026 15687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9f650396c2d412b8ebbb0b661598399: Generated new TSK 0
I20260811 09:01:38.072986 15687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:01:38.096138 15679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:38.088482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.9.40.126:40697)
0811 09:01:38.088714 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:38.088722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:38.089136 (+ 414us) server_negotiation.cc:407] Connection header received
0811 09:01:38.089363 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:38.089369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:38.089471 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:38.089668 (+ 197us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:01:38.090144 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.090939 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.092338 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.092844 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.093012 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:38.093599 (+ 587us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:01:38.095479 (+ 1880us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:01:38.095826 (+ 347us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46548: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 09:01:38.096294 15679 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46548: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 09:01:38.096761 15710 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:38.088344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40697 (local address 127.0.0.1:46548)
0811 09:01:38.089023 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:38.089040 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:38.089168 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:38.089679 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:38.089682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:38.089692 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:01:38.089974 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:38.089981 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.091114 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.091119 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:38.092173 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:38.092185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.093091 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.093099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:38.093260 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:38.093309 (+ 49us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:01:38.095698 (+ 2389us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:01:38.095938 (+ 240us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:40697: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":375,"threads_started":1}
I20260811 09:01:38.097822 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15601
I20260811 09:01:38.099025 15671 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 09:01:38.108763 15711 webserver.cc:533] Webserver: sslize failed:
E20260811 09:01:38.108892 15711 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:01:38.191785 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15601
2026-08-11T09:01:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (754 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T09:01:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:38Z Disabled control of system clock
I20260811 09:01:38.286232 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39665
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:43539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39665
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/testchainca.cert with env {}
W20260811 09:01:38.627353 15716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:38.627667 15716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:38.627846 15716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:38.634995 15716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:38.635100 15716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:38.635193 15716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:38.635298 15716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:38.635360 15716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:01:38.635480 15716 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:01:38.635528 15716 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 09:01:38.645617 15716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:38.646972 15716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:38.648276 15716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:38.659848 15723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:38.660000 15724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:38.660174 15726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:38.660842 15716 server_base.cc:1034] running on GCE node
I20260811 09:01:38.661616 15716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:38.663023 15716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:38.664260 15716 hybrid_clock.cc:648] HybridClock initialized: now 1786438898664227 us; error 66 us; skew 500 ppm
I20260811 09:01:38.664628 15716 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:01:38.666908 15716 webserver.cc:460] Webserver started at http://127.9.40.126:36117/ using document root <none> and password file <none>
I20260811 09:01:38.667606 15716 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:38.667745 15716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:38.668076 15716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:38.671053 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "04749f65b473435da5b2eda17b7abf48"
format_stamp: "Formatted at 2026-08-11 09:01:38 on dist-test-slave-cbsf"
server_key: "942f493ab3780770b2c357df831245ba"
server_key_iv: "84f4e13ecd9c5138af6c882eaefa2bac"
server_key_version: "encryptionkey@0"
I20260811 09:01:38.671875 15716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "04749f65b473435da5b2eda17b7abf48"
format_stamp: "Formatted at 2026-08-11 09:01:38 on dist-test-slave-cbsf"
server_key: "942f493ab3780770b2c357df831245ba"
server_key_iv: "84f4e13ecd9c5138af6c882eaefa2bac"
server_key_version: "encryptionkey@0"
I20260811 09:01:38.677218 15716 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:01:38.680639 15736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:38.682009 15716 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:01:38.682257 15716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "04749f65b473435da5b2eda17b7abf48"
format_stamp: "Formatted at 2026-08-11 09:01:38 on dist-test-slave-cbsf"
server_key: "942f493ab3780770b2c357df831245ba"
server_key_iv: "84f4e13ecd9c5138af6c882eaefa2bac"
server_key_version: "encryptionkey@0"
I20260811 09:01:38.682495 15716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:38.708350 15716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:38.709383 15716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:38.709719 15716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:38.729408 15716 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39665
I20260811 09:01:38.729439 15793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39665 every 8 connection(s)
I20260811 09:01:38.730907 15716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:38.734786 15794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:38.738472 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15716
I20260811 09:01:38.738706 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:38.739529 9377 external_mini_cluster.cc:1428] Setting key be05631099522d5a98e97df5a9386f90
I20260811 09:01:38.747969 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48: Bootstrap starting.
I20260811 09:01:38.751287 15794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:38.752455 15794 log.cc:824] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:38.753559 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:38.741718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44602 (local address 127.9.40.126:39665)
0811 09:01:38.742678 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:38.742690 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:38.742730 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:01:38.744599 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:38.744622 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:38.745139 (+ 517us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:38.745564 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:38.746183 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.747465 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.748885 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.749601 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.749820 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:38.751034 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:38.751041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:38.751056 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:01:38.751650 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:38.751760 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:01:38.751788 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:38.751929 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:38.752170 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 09:01:38.752485 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":334,"threads_started":1}
I20260811 09:01:38.755498 15794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48: No bootstrap required, opened a new log
I20260811 09:01:38.757568 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:01:38.764995 15794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } }
I20260811 09:01:38.765419 15794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:38.765587 15794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04749f65b473435da5b2eda17b7abf48, State: Initialized, Role: FOLLOWER
I20260811 09:01:38.766291 15794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } }
I20260811 09:01:38.766537 15794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:38.766701 15794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:38.766896 15794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:38.769393 15794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } }
I20260811 09:01:38.769934 15794 leader_election.cc:302] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04749f65b473435da5b2eda17b7abf48; no voters:
I20260811 09:01:38.770612 15794 leader_election.cc:288] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:38.770810 15802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:38.772004 15802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [term 1 LEADER]: Becoming Leader. State: Replica: 04749f65b473435da5b2eda17b7abf48, State: Running, Role: LEADER
I20260811 09:01:38.772583 15802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } }
I20260811 09:01:38.773021 15794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:38.775687 15803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04749f65b473435da5b2eda17b7abf48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } } }
I20260811 09:01:38.776024 15803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:38.778275 15804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04749f65b473435da5b2eda17b7abf48. Latest consensus state: current_term: 1 leader_uuid: "04749f65b473435da5b2eda17b7abf48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04749f65b473435da5b2eda17b7abf48" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39665 } } }
I20260811 09:01:38.778600 15804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:38.779801 15810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:38.785475 15810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:38.795351 15810 catalog_manager.cc:1347] Generated new cluster ID: 0ad9b855762b4311a12994282b7013bb
I20260811 09:01:38.795495 15810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:38.810811 15810 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:38.812147 15810 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:38.817898 15810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04749f65b473435da5b2eda17b7abf48: Generated new TSK 0
I20260811 09:01:38.819005 15810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:01:38.950903 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:38.843300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.9.40.126:39665)
0811 09:01:38.843502 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:38.843509 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:38.843985 (+ 476us) server_negotiation.cc:407] Connection header received
0811 09:01:38.844198 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:38.844204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:38.844299 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:38.844496 (+ 197us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:01:38.845045 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.846233 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.847643 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.848372 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.848505 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:38.849078 (+ 573us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:01:38.950121 (+101043us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:01:38.950575 (+ 454us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44604: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 09:01:38.951063 15799 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44604: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 09:01:38.951247 15829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:38.843129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39665 (local address 127.0.0.1:44604)
0811 09:01:38.843870 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:38.843886 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:38.844011 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:38.844511 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:38.844515 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:38.844525 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:01:38.844878 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:38.844888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.846382 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.846386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:38.847498 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:38.847508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:38.848545 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:38.848549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:38.848677 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:38.848773 (+ 96us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:01:38.950481 (+101708us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:01:38.950788 (+ 307us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:39665: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":456,"threads_started":1}
I20260811 09:01:38.952597 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15716
I20260811 09:01:38.953987 15788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:39.066879 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15716
2026-08-11T09:01:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (852 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (447 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (464 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:40 dist-test-slave-cbsf krb5kdc[15847](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:40 dist-test-slave-cbsf krb5kdc[15847](info): set up 2 sockets
Aug 11 09:01:40 dist-test-slave-cbsf krb5kdc[15847](info): commencing operation
krb5kdc: starting...
W20260811 09:01:41.477424 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.458s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:41 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:01:41.765367 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36933
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36933
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:42.134290 15863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:42.134719 15863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:42.134898 15863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:42.142931 15863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:42.143158 15863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:42.143280 15863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:42.143392 15863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:42.143488 15863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:42.154381 15863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36933
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:42.155936 15863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:42.157444 15863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:42.170604 15868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:42.170580 15869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:42.170660 15871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:42.171348 15863 server_base.cc:1034] running on GCE node
I20260811 09:01:42.172302 15863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:42.174026 15863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:42.175266 15863 hybrid_clock.cc:648] HybridClock initialized: now 1786438902175198 us; error 58 us; skew 500 ppm
I20260811 09:01:42.175637 15863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:42 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438902, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:42.183105 15863 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:01:42.184943 15863 webserver.cc:460] Webserver started at http://127.9.40.126:45099/ using document root <none> and password file <none>
I20260811 09:01:42.185696 15863 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:42.185808 15863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:42.186093 15863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:42.189478 15863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "5345025c6c994af5af6b91ba103c119e"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "3fb6faf81b4d78ce075c58dcd702358b"
server_key_iv: "f61de06ae82391bb57b020988f43502b"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.190310 15863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "5345025c6c994af5af6b91ba103c119e"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "3fb6faf81b4d78ce075c58dcd702358b"
server_key_iv: "f61de06ae82391bb57b020988f43502b"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.196228 15863 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:01:42.200201 15881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:42.201845 15863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:42.202050 15863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "5345025c6c994af5af6b91ba103c119e"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "3fb6faf81b4d78ce075c58dcd702358b"
server_key_iv: "f61de06ae82391bb57b020988f43502b"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.202251 15863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:42.219485 15863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:42.224345 15863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:42.224632 15863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:42.245672 15863 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36933
I20260811 09:01:42.245697 15938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36933 every 8 connection(s)
I20260811 09:01:42.247242 15863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:42.252297 15939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:42.254150 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15863
I20260811 09:01:42.254405 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:42.255067 9377 external_mini_cluster.cc:1428] Setting key 159cd0d2316752e42d7672f6fd281fa1
I20260811 09:01:42.267279 15939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e: Bootstrap starting.
Aug 11 09:01:42 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:42.271047 15939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:42.272260 15939 log.cc:824] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:42.275722 15939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e: No bootstrap required, opened a new log
I20260811 09:01:42.280289 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:42.257486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.9.40.126:36933)
0811 09:01:42.258425 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:42.258450 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:42.258503 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:01:42.260371 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:42.260403 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:42.261013 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:42.261514 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:42.263150 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.264576 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.266025 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.266785 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.267024 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:42.271706 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:42.271717 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:42.271735 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:42.271785 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:42.275249 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:42.276235 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:42.276243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:42.276250 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:42.276455 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:42.277017 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:42.277023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:42.277031 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:42.277700 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:42.278064 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:42.278596 (+ 532us) server_negotiation.cc:299] Negotiation successful
0811 09:01:42.279098 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":414,"threads_started":1}
I20260811 09:01:42.284570 15939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } }
I20260811 09:01:42.285094 15939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:42.285211 15939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5345025c6c994af5af6b91ba103c119e, State: Initialized, Role: FOLLOWER
I20260811 09:01:42.285817 15939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } }
I20260811 09:01:42.286084 15939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:42.286242 15939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:42.286425 15939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:42.289026 15939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } }
I20260811 09:01:42.289711 15939 leader_election.cc:302] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5345025c6c994af5af6b91ba103c119e; no voters:
I20260811 09:01:42.290508 15939 leader_election.cc:288] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:42.290635 15944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:42.291435 15944 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [term 1 LEADER]: Becoming Leader. State: Replica: 5345025c6c994af5af6b91ba103c119e, State: Running, Role: LEADER
I20260811 09:01:42.292109 15944 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } }
I20260811 09:01:42.293169 15939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:42.295418 15948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5345025c6c994af5af6b91ba103c119e. Latest consensus state: current_term: 1 leader_uuid: "5345025c6c994af5af6b91ba103c119e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } } }
I20260811 09:01:42.295897 15948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:42.296365 15946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5345025c6c994af5af6b91ba103c119e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5345025c6c994af5af6b91ba103c119e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36933 } } }
I20260811 09:01:42.296759 15946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:42.298214 15955 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:42.303871 15955 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:42.315141 15955 catalog_manager.cc:1347] Generated new cluster ID: 82a1eff9b4b6424abf0385d56a73c5d4
I20260811 09:01:42.315356 15955 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:42.335723 15955 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:42.337548 15955 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:42.352663 15955 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5345025c6c994af5af6b91ba103c119e: Generated new TSK 0
I20260811 09:01:42.354059 15955 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:01:42.439716 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:42.829525 15970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:42.829872 15970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:42.830084 15970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:42.837832 15970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:42.838016 15970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:42.838235 15970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:42.848647 15970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:42.850257 15970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:42.851752 15970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:42.867375 15975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:42.867301 15976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:42.867375 15980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:42.868364 15970 server_base.cc:1034] running on GCE node
I20260811 09:01:42.869230 15970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:42.870508 15970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:42.874044 15970 hybrid_clock.cc:648] HybridClock initialized: now 1786438902873922 us; error 103 us; skew 500 ppm
I20260811 09:01:42.874610 15970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:42 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438902, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:42.878724 15970 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:01:42.880672 15970 webserver.cc:460] Webserver started at http://127.9.40.65:44441/ using document root <none> and password file <none>
I20260811 09:01:42.881603 15970 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:42.881778 15970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:42.882143 15970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:42.885654 15970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "563f6af22f0e46a1a131aab28871a07f"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "7d474f3628a26cd38e05c628e5f280ad"
server_key_iv: "51855bf73f3dae087353c796c9b5bc77"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.886684 15970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "563f6af22f0e46a1a131aab28871a07f"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "7d474f3628a26cd38e05c628e5f280ad"
server_key_iv: "51855bf73f3dae087353c796c9b5bc77"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.893379 15970 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 09:01:42.898052 15988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:42.899865 15970 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:01:42.900117 15970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "563f6af22f0e46a1a131aab28871a07f"
format_stamp: "Formatted at 2026-08-11 09:01:42 on dist-test-slave-cbsf"
server_key: "7d474f3628a26cd38e05c628e5f280ad"
server_key_iv: "51855bf73f3dae087353c796c9b5bc77"
server_key_version: "encryptionkey@0"
I20260811 09:01:42.900391 15970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:42.922715 15970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:42.927975 15970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:42.928361 15970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:42.929657 15970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:42.931615 15970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:42.931753 15970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:42.931926 15970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:42.932004 15970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:42.970130 15970 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33993
I20260811 09:01:42.970177 16127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33993 every 8 connection(s)
I20260811 09:01:42.971974 15970 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:42.974051 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 15970
I20260811 09:01:42.974318 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:42.975095 9377 external_mini_cluster.cc:1428] Setting key 576d651c028846f9a42fec02cfd8aa87
Aug 11 09:01:42 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438902, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:42.998253 16131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:42.975356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53649 (local address 127.9.40.126:36933)
0811 09:01:42.975869 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:42.975874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:42.977601 (+ 1727us) server_negotiation.cc:407] Connection header received
0811 09:01:42.979201 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:42.979207 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:42.979307 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:42.979437 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:42.981213 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.981949 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.983635 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.984074 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.984227 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:42.988547 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:42.988558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:42.988567 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:42.988624 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:42.992535 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:42.994526 (+ 1991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:42.994535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:42.994542 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:42.994728 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:42.995588 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:42.995596 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:42.995603 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:42.996103 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:42.996277 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:42.997521 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0811 09:01:42.997833 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":245,"threads_started":1}
I20260811 09:01:42.999825 16132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:42.976030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36933 (local address 127.9.40.65:53649)
0811 09:01:42.977040 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:42.977107 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:42.978874 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:42.979711 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:42.979721 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:42.980295 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:42.981018 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:42.981038 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.982141 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.982146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:42.983458 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:42.983472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:42.984356 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:42.984364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:42.984623 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:42.985458 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:42.985488 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:42.988196 (+ 2708us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:42.992849 (+ 4653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:42.992865 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:42.992952 (+ 87us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:42.994309 (+ 1357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:42.994885 (+ 576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:42.994891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:42.994897 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:42.995388 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:42.996282 (+ 894us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:42.996292 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:42.996942 (+ 650us) client_negotiation.cc:769] Sending connection context
0811 09:01:42.997366 (+ 424us) client_negotiation.cc:240] Negotiation successful
0811 09:01:42.998041 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":432,"threads_started":1}
I20260811 09:01:43.001786 16128 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36933
I20260811 09:01:43.002238 16128 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:43.003413 16128 heartbeater.cc:511] Master 127.9.40.126:36933 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:01:43.006445 15900 ts_manager.cc:194] Registered new tserver with Master: 563f6af22f0e46a1a131aab28871a07f (127.9.40.65:33993)
I20260811 09:01:43.008124 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:53649
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:01:43.041131 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:43.427368 16135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:43.427721 16135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:43.427824 16135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:43.435796 16135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:43.435945 16135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:43.436071 16135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:43.449038 16135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:43.450644 16135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:43.452103 16135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:43.467975 16145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:43.468151 16142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:43.468155 16143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:43.469434 16135 server_base.cc:1034] running on GCE node
I20260811 09:01:43.470402 16135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:43.471693 16135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:43.472975 16135 hybrid_clock.cc:648] HybridClock initialized: now 1786438903472925 us; error 79 us; skew 500 ppm
I20260811 09:01:43.473380 16135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:43 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438903, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:43.477180 16135 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:01:43.479008 16135 webserver.cc:460] Webserver started at http://127.9.40.66:36751/ using document root <none> and password file <none>
I20260811 09:01:43.479776 16135 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:43.479895 16135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:43.480252 16135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:43.483788 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "f2646d2195f94ff8870edb00e804f7cb"
format_stamp: "Formatted at 2026-08-11 09:01:43 on dist-test-slave-cbsf"
server_key: "7b38659c7e11919696ed313304d51c5f"
server_key_iv: "9c73e9aca6a64270aacd98a462f82519"
server_key_version: "encryptionkey@0"
I20260811 09:01:43.484858 16135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "f2646d2195f94ff8870edb00e804f7cb"
format_stamp: "Formatted at 2026-08-11 09:01:43 on dist-test-slave-cbsf"
server_key: "7b38659c7e11919696ed313304d51c5f"
server_key_iv: "9c73e9aca6a64270aacd98a462f82519"
server_key_version: "encryptionkey@0"
I20260811 09:01:43.491811 16135 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260811 09:01:43.496342 16152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:43.498267 16135 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:43.498512 16135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "f2646d2195f94ff8870edb00e804f7cb"
format_stamp: "Formatted at 2026-08-11 09:01:43 on dist-test-slave-cbsf"
server_key: "7b38659c7e11919696ed313304d51c5f"
server_key_iv: "9c73e9aca6a64270aacd98a462f82519"
server_key_version: "encryptionkey@0"
I20260811 09:01:43.498772 16135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:43.519171 16135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:43.525300 16135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:43.525629 16135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:43.527034 16135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:43.528854 16135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:43.528950 16135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:43.529083 16135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:43.529135 16135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:43.569032 16135 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35703
I20260811 09:01:43.569067 16281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35703 every 8 connection(s)
I20260811 09:01:43.570668 16135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:43.572991 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16135
I20260811 09:01:43.573220 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:43.573705 9377 external_mini_cluster.cc:1428] Setting key 51124fb6543bbbbcbcc71b192eff3675
Aug 11 09:01:43 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438903, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:43.597991 16284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:43.574202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50891 (local address 127.9.40.126:36933)
0811 09:01:43.574743 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:43.574750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:43.576087 (+ 1337us) server_negotiation.cc:407] Connection header received
0811 09:01:43.578251 (+ 2164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:43.578260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:43.578378 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:43.578531 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:43.580497 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:43.581864 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:43.583922 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:43.584835 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:43.585017 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:43.590453 (+ 5436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:43.590473 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:43.590479 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:43.590537 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:43.593861 (+ 3324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:43.594923 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:43.594930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:43.594933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:43.595094 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:43.595612 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:43.595618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:43.595621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:43.595953 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:43.596123 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:43.597072 (+ 949us) server_negotiation.cc:299] Negotiation successful
0811 09:01:43.597404 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":269,"threads_started":1}
I20260811 09:01:43.598835 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:43.574870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36933 (local address 127.9.40.66:50891)
0811 09:01:43.575872 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:43.575933 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:43.577871 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:43.578882 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:43.578893 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:43.579534 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:43.580240 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:43.580261 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:43.582134 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:43.582140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:43.583687 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:43.583707 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:43.585148 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:43.585155 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:43.585410 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:43.586329 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:43.586372 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:43.589654 (+ 3282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:43.594086 (+ 4432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:43.594098 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:43.594127 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:43.594763 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:43.595238 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:43.595243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:43.595247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:43.595482 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:43.596109 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:43.596119 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:43.596536 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 09:01:43.596919 (+ 383us) client_negotiation.cc:240] Negotiation successful
0811 09:01:43.597477 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":381,"threads_started":1}
I20260811 09:01:43.600759 16282 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36933
I20260811 09:01:43.601181 16282 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260811 09:01:43.602357 16282 heartbeater.cc:511] Master 127.9.40.126:36933 requested a full tablet report, sending...
I20260811 09:01:43.604431 15900 ts_manager.cc:194] Registered new tserver with Master: f2646d2195f94ff8870edb00e804f7cb (127.9.40.66:35703)
I20260811 09:01:43.605674 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:50891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:01:43.637701 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37711
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:44.011003 16289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:44.011400 16289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:44.011567 16289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 09:01:44.011729 16128 heartbeater.cc:503] Master 127.9.40.126:36933 was elected leader, sending a full tablet report...
W20260811 09:01:44.020212 16289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:44.020370 16289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:44.020490 16289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:44.032038 16289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37711
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36933
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:44.033676 16289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:44.035279 16289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:44.051023 16296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:44.051082 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:44.050993 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:44.051879 16289 server_base.cc:1034] running on GCE node
I20260811 09:01:44.052594 16289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:44.053741 16289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:44.054986 16289 hybrid_clock.cc:648] HybridClock initialized: now 1786438904054909 us; error 69 us; skew 500 ppm
I20260811 09:01:44.055461 16289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.059705 16289 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:01:44.061801 16289 webserver.cc:460] Webserver started at http://127.9.40.67:46843/ using document root <none> and password file <none>
I20260811 09:01:44.062767 16289 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:44.062970 16289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:44.063360 16289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:44.067159 16289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "2f5a131d88374e3da5ba0ebcd1d77f23"
format_stamp: "Formatted at 2026-08-11 09:01:44 on dist-test-slave-cbsf"
server_key: "2d67f524e54d9824a323c19c056ee82f"
server_key_iv: "0dfb70a28d2f3e166c6627e12cf9d4c6"
server_key_version: "encryptionkey@0"
I20260811 09:01:44.068276 16289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "2f5a131d88374e3da5ba0ebcd1d77f23"
format_stamp: "Formatted at 2026-08-11 09:01:44 on dist-test-slave-cbsf"
server_key: "2d67f524e54d9824a323c19c056ee82f"
server_key_iv: "0dfb70a28d2f3e166c6627e12cf9d4c6"
server_key_version: "encryptionkey@0"
I20260811 09:01:44.074800 16289 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:01:44.079186 16307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:44.080904 16289 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:44.081139 16289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "2f5a131d88374e3da5ba0ebcd1d77f23"
format_stamp: "Formatted at 2026-08-11 09:01:44 on dist-test-slave-cbsf"
server_key: "2d67f524e54d9824a323c19c056ee82f"
server_key_iv: "0dfb70a28d2f3e166c6627e12cf9d4c6"
server_key_version: "encryptionkey@0"
I20260811 09:01:44.081411 16289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:44.099251 16289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:44.104501 16289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:44.104964 16289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:44.106333 16289 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:44.108038 16289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:44.108170 16289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:44.108326 16289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:44.108408 16289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:44.146061 16289 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:41801
I20260811 09:01:44.146102 16446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:41801 every 8 connection(s)
I20260811 09:01:44.147774 16289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:44.157166 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16289
I20260811 09:01:44.157401 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:44.157948 9377 external_mini_cluster.cc:1428] Setting key 074ddf0ecf67b20e8909ebb62f44c205
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:01:44.171082 16449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.151016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:39653 (local address 127.9.40.126:36933)
0811 09:01:44.151548 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.151553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.152613 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 09:01:44.154688 (+ 2075us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.154695 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.154813 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.154948 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.156783 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.157824 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.159881 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.160336 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.160494 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.164408 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.164413 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.164416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.164457 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.167435 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.168361 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.168367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.168371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.168518 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.169083 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.169089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.169092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.169400 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.169584 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.170390 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.170665 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":293,"threads_started":1}
I20260811 09:01:44.172152 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.151447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36933 (local address 127.9.40.67:39653)
0811 09:01:44.152400 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:44.152443 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:44.154346 (+ 1903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:44.155230 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:44.155240 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:44.155844 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:44.156518 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:44.156539 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.158083 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.158088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:44.159722 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:44.159737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.160598 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.160608 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:44.160923 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.161681 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:44.161706 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:44.164115 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:44.167653 (+ 3538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:44.167666 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:44.167694 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:44.168215 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:44.168721 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:44.168727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:44.168731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:44.168957 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:44.169544 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:44.169554 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:44.169977 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 09:01:44.170271 (+ 294us) client_negotiation.cc:240] Negotiation successful
0811 09:01:44.170878 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":391,"threads_started":1}
I20260811 09:01:44.174160 16447 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36933
I20260811 09:01:44.174551 16447 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:44.175434 16447 heartbeater.cc:511] Master 127.9.40.126:36933 requested a full tablet report, sending...
I20260811 09:01:44.177165 15900 ts_manager.cc:194] Registered new tserver with Master: 2f5a131d88374e3da5ba0ebcd1d77f23 (127.9.40.67:41801)
I20260811 09:01:44.177985 15900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:39653
I20260811 09:01:44.187319 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/test-admin.keytab.
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.223520 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 09:01:44.229879 16455 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:44.226787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34300 (local address 127.9.40.65:33993)
0811 09:01:44.227578 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.227593 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.227619 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:01:44.227685 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.227691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.227916 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.228108 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.229442 (+ 1334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34300: BlockingRecv error: recv got EOF from 127.0.0.1:34300 (error 108)
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":401,"threads_started":1}
W20260811 09:01:44.230286 9377 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.232177 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.233805 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:44.252596 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.235934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.9.40.65:33993)
0811 09:01:44.236118 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.236123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.236390 (+ 267us) server_negotiation.cc:407] Connection header received
0811 09:01:44.236534 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.236544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.236617 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.236790 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.237918 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.238945 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.240360 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.241195 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.241321 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.245981 (+ 4660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.245998 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.246016 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.246063 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.249336 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.250248 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.250257 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.250260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.250413 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.250842 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.250847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.250855 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.251264 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.251582 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.251973 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.252284 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:01:44.259604 16455 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:44.257431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34326 (local address 127.9.40.65:33993)
0811 09:01:44.257674 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.257682 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.257967 (+ 285us) server_negotiation.cc:407] Connection header received
0811 09:01:44.258211 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.258216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.258286 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.258397 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.259282 (+ 885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34326: BlockingRecv error: recv got EOF from 127.0.0.1:34326 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 09:01:44.260288 9377 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc)
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.262225 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.263916 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:44.280328 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.265640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34338 (local address 127.9.40.65:33993)
0811 09:01:44.265891 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.265898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.266211 (+ 313us) server_negotiation.cc:407] Connection header received
0811 09:01:44.266431 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.266435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.266503 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.266612 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.267731 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.268463 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.269844 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.270339 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.270508 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.274717 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.274724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.274728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.274799 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.277754 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.278581 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.278587 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.278590 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.278741 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.279108 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.279111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.279113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.279394 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.279605 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.279885 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.280072 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 09:01:44.285913 16455 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:01:44.283763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.9.40.65:33993)
0811 09:01:44.283989 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.283996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.284168 (+ 172us) server_negotiation.cc:407] Connection header received
0811 09:01:44.284379 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.284388 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.284499 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.284614 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.285635 (+ 1021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34354: BlockingRecv error: recv got EOF from 127.0.0.1:34354 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 09:01:44.286063 9377 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786438852478101-9377-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.287990 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:44.289805 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:01:44.307536 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.291516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.9.40.65:33993)
0811 09:01:44.291803 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.291813 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.292082 (+ 269us) server_negotiation.cc:407] Connection header received
0811 09:01:44.292325 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.292330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.292397 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.292507 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.293739 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.294566 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.295916 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.296355 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.296500 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.301021 (+ 4521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.301028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.301031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.301079 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.304085 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.305143 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.305148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.305150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.305386 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.305826 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.305832 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.305840 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.306228 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.306598 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.306920 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.307190 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 09:01:44.309383 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15970
I20260811 09:01:44.323632 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.310415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.9.40.65:33993)
0811 09:01:44.310639 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.310648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.311005 (+ 357us) server_negotiation.cc:407] Connection header received
0811 09:01:44.311235 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.311240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.311304 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.311404 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.312470 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.313268 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.314525 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.314992 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.315146 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.317889 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.317895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.317899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.317941 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.320538 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.321417 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.321423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.321433 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.321596 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.322031 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.322038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.322041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.322349 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.322669 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.323025 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.323291 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:44.324365 16120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:44.413679 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15970
I20260811 09:01:44.436436 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16135
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:01:44.454591 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.437550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.9.40.66:35703)
0811 09:01:44.438210 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.438218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.438248 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:01:44.438318 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.438324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.438553 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.438700 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.439762 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.440998 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.442589 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.443228 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.443344 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.448040 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.448060 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.448079 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.448139 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.451428 (+ 3289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.452394 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.452402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.452404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.452550 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.453008 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.453012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.453014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.453432 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.453686 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.454047 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.454281 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":329,"threads_started":1}
I20260811 09:01:44.456175 16276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:44.545982 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16135
I20260811 09:01:44.566643 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16289
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[15847](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:01:44.585367 16474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:44.568067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49606 (local address 127.9.40.67:41801)
0811 09:01:44.568903 (+ 836us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:44.568911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:44.568947 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:01:44.569011 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:44.569019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:44.569272 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:44.569420 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:44.570681 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.571748 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.573429 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:44.574097 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:44.574225 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:44.578686 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:44.578701 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:44.578719 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:44.578763 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:44.582139 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.583116 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.583125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.583128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.583287 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:44.583699 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:44.583702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:44.583704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:44.584117 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:44.584380 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:44.584663 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 09:01:44.585029 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":433,"threads_started":1}
I20260811 09:01:44.587009 16441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:44.682734 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16289
I20260811 09:01:44.704406 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 15863
I20260811 09:01:44.706286 15933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:44.817224 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 15863
2026-08-11T09:01:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4857 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[16481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[16481](info): set up 2 sockets
Aug 11 09:01:44 dist-test-slave-cbsf krb5kdc[16481](info): commencing operation
krb5kdc: starting...
W20260811 09:01:46.034399 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:46 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
I20260811 09:01:46.333539 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45611
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:46.717331 16498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:46.717698 16498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:46.717868 16498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:46.726047 16498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:46.726197 16498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:46.726297 16498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:46.726411 16498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:46.726492 16498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:46.737658 16498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:46.739229 16498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:46.740830 16498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:46.754854 16503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:46.754689 16504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:46.754729 16506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:46.755931 16498 server_base.cc:1034] running on GCE node
I20260811 09:01:46.757350 16498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:46.759080 16498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:46.760331 16498 hybrid_clock.cc:648] HybridClock initialized: now 1786438906760268 us; error 54 us; skew 500 ppm
I20260811 09:01:46.760854 16498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:46 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:46.765162 16498 init.cc:377] Logged in from keytab as oryx/127.9.40.126@KRBTEST.COM (short username oryx)
I20260811 09:01:46.767159 16498 webserver.cc:460] Webserver started at http://127.9.40.126:42195/ using document root <none> and password file <none>
I20260811 09:01:46.768026 16498 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:46.768200 16498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:46.768561 16498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:46.772167 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2"
format_stamp: "Formatted at 2026-08-11 09:01:46 on dist-test-slave-cbsf"
server_key: "b73fc821fde6dc0081f57c900aa96c96"
server_key_iv: "4007822b2664decc558b68b3ac09229d"
server_key_version: "encryptionkey@0"
I20260811 09:01:46.773254 16498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2"
format_stamp: "Formatted at 2026-08-11 09:01:46 on dist-test-slave-cbsf"
server_key: "b73fc821fde6dc0081f57c900aa96c96"
server_key_iv: "4007822b2664decc558b68b3ac09229d"
server_key_version: "encryptionkey@0"
I20260811 09:01:46.779632 16498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:01:46.783979 16517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:46.785652 16498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:46.785879 16498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2"
format_stamp: "Formatted at 2026-08-11 09:01:46 on dist-test-slave-cbsf"
server_key: "b73fc821fde6dc0081f57c900aa96c96"
server_key_iv: "4007822b2664decc558b68b3ac09229d"
server_key_version: "encryptionkey@0"
I20260811 09:01:46.786113 16498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:46.802100 16498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:46.806948 16498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:46.807291 16498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:46.828070 16498 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45611
I20260811 09:01:46.828105 16574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45611 every 8 connection(s)
I20260811 09:01:46.829668 16498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:46.833954 16575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:46.834515 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16498
I20260811 09:01:46.834779 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:46.835506 9377 external_mini_cluster.cc:1428] Setting key 9d15e20bd7ccf62aabdf56ba208346bc
I20260811 09:01:46.850523 16575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Bootstrap starting.
Aug 11 09:01:46 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:46.854339 16575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:46.855911 16575 log.cc:824] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:46.859562 16575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: No bootstrap required, opened a new log
I20260811 09:01:46.864910 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:46.838007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.9.40.126:45611)
0811 09:01:46.838978 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:46.839008 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:46.839080 (+ 72us) server_negotiation.cc:407] Connection header received
0811 09:01:46.841665 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:46.841708 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:46.842421 (+ 713us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:46.842993 (+ 572us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:46.844579 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:46.846790 (+ 2211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:46.848239 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:46.849483 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:46.849684 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:46.854956 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:46.854972 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:46.854999 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:46.855059 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:46.859121 (+ 4062us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:46.860230 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:46.860240 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:46.860248 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:46.860471 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:46.860997 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:46.861007 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:46.861010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:46.861707 (+ 697us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:46.862108 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:46.862643 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 09:01:46.863270 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":386,"threads_started":1}
I20260811 09:01:46.869953 16575 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } }
I20260811 09:01:46.870452 16575 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:46.870569 16575 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38f97f9a1bfa447ea1f8a39f791b7ee2, State: Initialized, Role: FOLLOWER
I20260811 09:01:46.871251 16575 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } }
I20260811 09:01:46.871598 16575 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:46.871866 16575 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:46.872165 16575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:46.874946 16575 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } }
I20260811 09:01:46.876101 16575 leader_election.cc:302] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38f97f9a1bfa447ea1f8a39f791b7ee2; no voters:
I20260811 09:01:46.876960 16575 leader_election.cc:288] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:46.877156 16580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:46.878103 16580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 LEADER]: Becoming Leader. State: Replica: 38f97f9a1bfa447ea1f8a39f791b7ee2, State: Running, Role: LEADER
I20260811 09:01:46.878769 16580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } }
I20260811 09:01:46.879657 16575 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:46.882704 16581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } }
I20260811 09:01:46.883327 16581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:46.882678 16582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f97f9a1bfa447ea1f8a39f791b7ee2. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } }
I20260811 09:01:46.883800 16582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:46.889005 16588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:46.894774 16588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:46.904999 16588 catalog_manager.cc:1347] Generated new cluster ID: 49232210f17d47eb9a822f81a921a51e
I20260811 09:01:46.905157 16588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:46.925482 16588 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:46.926976 16588 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:46.937242 16588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Generated new TSK 0
I20260811 09:01:46.938285 16588 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
I20260811 09:01:47.015901 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:47.428782 16602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:47.429127 16602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:47.429234 16602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:47.437206 16602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:47.437342 16602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:47.437532 16602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:47.448493 16602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:47.450181 16602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:47.451773 16602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:47.466764 16610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:47.466792 16612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:47.466871 16608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:47.468022 16602 server_base.cc:1034] running on GCE node
I20260811 09:01:47.468850 16602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:47.470059 16602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:47.471303 16602 hybrid_clock.cc:648] HybridClock initialized: now 1786438907471230 us; error 71 us; skew 500 ppm
I20260811 09:01:47.471698 16602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:47 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438907, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:47.479440 16602 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260811 09:01:47.481598 16602 webserver.cc:460] Webserver started at http://127.9.40.65:38649/ using document root <none> and password file <none>
I20260811 09:01:47.482533 16602 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:47.482688 16602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:47.483004 16602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:47.486888 16602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "bcf48d96894746e08e1ca33b78bea683"
format_stamp: "Formatted at 2026-08-11 09:01:47 on dist-test-slave-cbsf"
server_key: "6d536535dc4bc6d049fb65c7956af196"
server_key_iv: "668982991ce68e212c1f842876af6801"
server_key_version: "encryptionkey@0"
I20260811 09:01:47.487915 16602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "bcf48d96894746e08e1ca33b78bea683"
format_stamp: "Formatted at 2026-08-11 09:01:47 on dist-test-slave-cbsf"
server_key: "6d536535dc4bc6d049fb65c7956af196"
server_key_iv: "668982991ce68e212c1f842876af6801"
server_key_version: "encryptionkey@0"
I20260811 09:01:47.495792 16602 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 09:01:47.501025 16621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:47.503281 16602 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.002s
I20260811 09:01:47.503585 16602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "bcf48d96894746e08e1ca33b78bea683"
format_stamp: "Formatted at 2026-08-11 09:01:47 on dist-test-slave-cbsf"
server_key: "6d536535dc4bc6d049fb65c7956af196"
server_key_iv: "668982991ce68e212c1f842876af6801"
server_key_version: "encryptionkey@0"
I20260811 09:01:47.503850 16602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:47.521540 16602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:47.526914 16602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:47.527304 16602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:47.528530 16602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:47.530323 16602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:47.530417 16602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:47.530606 16602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:47.530666 16602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:47.571142 16602 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42497
I20260811 09:01:47.571195 16749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42497 every 8 connection(s)
I20260811 09:01:47.572815 16602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:47.582264 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16602
I20260811 09:01:47.582553 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:47.583357 9377 external_mini_cluster.cc:1428] Setting key 47794f1ff661ecfa63d14fedbf40dbbc
Aug 11 09:01:47 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438907, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:47.600507 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:47.576028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:41879 (local address 127.9.40.126:45611)
0811 09:01:47.576595 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:47.576601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:47.578125 (+ 1524us) server_negotiation.cc:407] Connection header received
0811 09:01:47.580367 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:47.580374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:47.580490 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:47.580621 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:47.582971 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:47.583862 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:47.586088 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:47.586678 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:47.586819 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:47.592593 (+ 5774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:47.592600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:47.592604 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:47.592650 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:47.595782 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:47.597241 (+ 1459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:47.597252 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:47.597257 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:47.597481 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:47.598052 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:47.598060 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:47.598063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:47.598533 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:47.598768 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:47.599638 (+ 870us) server_negotiation.cc:299] Negotiation successful
0811 09:01:47.599971 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":287,"threads_started":1}
I20260811 09:01:47.601361 16753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:47.576859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:41879)
0811 09:01:47.577890 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:47.577946 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:47.579972 (+ 2026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:47.580964 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:47.580975 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:47.581617 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:47.582676 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:47.582713 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:47.584108 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:47.584113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:47.585848 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:47.585872 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:47.586992 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:47.587002 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:47.587443 (+ 441us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:47.588364 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:47.588406 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:47.592175 (+ 3769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:47.596086 (+ 3911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:47.596103 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:47.596130 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:47.597016 (+ 886us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:47.597636 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:47.597641 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:47.597643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:47.597893 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:47.598716 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:47.598724 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:47.599132 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 09:01:47.599414 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 09:01:47.599939 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":510,"threads_started":1}
I20260811 09:01:47.603302 16750 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:47.603689 16750 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:47.604539 16750 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:47.607139 16536 ts_manager.cc:194] Registered new tserver with Master: bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:47.609436 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:41879
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
I20260811 09:01:47.647965 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:48.025956 16758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:48.026363 16758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:48.026543 16758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:48.034319 16758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:48.034526 16758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:48.034729 16758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:48.045413 16758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:48.047017 16758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:48.048616 16758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:48.063902 16765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:48.063989 16768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:48.063915 16766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:48.064917 16758 server_base.cc:1034] running on GCE node
I20260811 09:01:48.065603 16758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:48.066681 16758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:48.067910 16758 hybrid_clock.cc:648] HybridClock initialized: now 1786438908067840 us; error 69 us; skew 500 ppm
I20260811 09:01:48.068262 16758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:48 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438908, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:48.071986 16758 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260811 09:01:48.073876 16758 webserver.cc:460] Webserver started at http://127.9.40.66:36587/ using document root <none> and password file <none>
I20260811 09:01:48.074743 16758 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:48.074913 16758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:48.075228 16758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:48.078770 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "72af8e72ace1481ea8a2f10a379b4729"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "64a2ff5bbb7707e6c628ea0954bbf225"
server_key_iv: "0f20c6dfde15778679b67d4028ec20b0"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.079671 16758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "72af8e72ace1481ea8a2f10a379b4729"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "64a2ff5bbb7707e6c628ea0954bbf225"
server_key_iv: "0f20c6dfde15778679b67d4028ec20b0"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.086112 16758 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:01:48.090610 16776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.092491 16758 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 09:01:48.092775 16758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "72af8e72ace1481ea8a2f10a379b4729"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "64a2ff5bbb7707e6c628ea0954bbf225"
server_key_iv: "0f20c6dfde15778679b67d4028ec20b0"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.093031 16758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:48.109275 16758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:48.114107 16758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:48.114461 16758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:48.115581 16758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:48.117388 16758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:48.117502 16758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.117660 16758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:48.117738 16758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.156181 16758 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46395
I20260811 09:01:48.156245 16904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46395 every 8 connection(s)
I20260811 09:01:48.157789 16758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:48.158548 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16758
I20260811 09:01:48.158802 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:48.159453 9377 external_mini_cluster.cc:1428] Setting key 4e88d571915d2dccec02c0237e91d80f
Aug 11 09:01:48 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438908, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:48.183568 16909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:48.161181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:40987 (local address 127.9.40.126:45611)
0811 09:01:48.161716 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:48.161724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:48.163169 (+ 1445us) server_negotiation.cc:407] Connection header received
0811 09:01:48.165509 (+ 2340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:48.165516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:48.165630 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:48.165772 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:48.167895 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.168782 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.170733 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.171286 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.171462 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:48.176504 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:48.176510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:48.176513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:48.176557 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:48.179644 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:48.180746 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:48.180753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:48.180757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:48.180937 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:48.181522 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:48.181529 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:48.181532 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:48.181852 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:48.182026 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:48.182856 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 09:01:48.183151 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":273,"threads_started":1}
I20260811 09:01:48.184854 16910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:48.161940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:40987)
0811 09:01:48.162930 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:48.162988 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:48.165095 (+ 2107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:48.166097 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:48.166109 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:48.166795 (+ 686us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:48.167619 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:48.167644 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.168996 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.169000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:48.170509 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:48.170528 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.171598 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.171606 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:48.171901 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:48.172845 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:48.172889 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:48.176153 (+ 3264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:48.179896 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:48.179909 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:48.179927 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:48.180566 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:48.181076 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:48.181082 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:48.181086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:48.181393 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:48.182012 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:48.182022 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:48.182452 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 09:01:48.182755 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 09:01:48.183386 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":436,"threads_started":1}
I20260811 09:01:48.186822 16907 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:48.187275 16907 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260811 09:01:48.188495 16907 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:48.190550 16536 ts_manager.cc:194] Registered new tserver with Master: 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
I20260811 09:01:48.191813 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:40987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260811 09:01:48.223953 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:48.602316 16913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:48.602629 16913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:48.602783 16913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:48.611075 16913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:48.611271 16913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:48.611464 16913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:48.613991 16750 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
I20260811 09:01:48.623440 16913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:48.625015 16913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:48.626588 16913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:48.641727 16919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:48.641702 16922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:48.641750 16920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:48.642738 16913 server_base.cc:1034] running on GCE node
I20260811 09:01:48.643496 16913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:48.644748 16913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:48.646015 16913 hybrid_clock.cc:648] HybridClock initialized: now 1786438908645963 us; error 64 us; skew 500 ppm
I20260811 09:01:48.646385 16913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:48 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438908, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:48.650127 16913 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260811 09:01:48.651983 16913 webserver.cc:460] Webserver started at http://127.9.40.67:35625/ using document root <none> and password file <none>
I20260811 09:01:48.652959 16913 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:48.653126 16913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:48.653479 16913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:48.656987 16913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "994ef811a0e34551a75344f6a17f7197"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "46d9a61f38d5acdb5a29084f6e23c889"
server_key_iv: "f7386e89bf501501f328e4e10b6640a6"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.657943 16913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "994ef811a0e34551a75344f6a17f7197"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "46d9a61f38d5acdb5a29084f6e23c889"
server_key_iv: "f7386e89bf501501f328e4e10b6640a6"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.664502 16913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:01:48.668993 16929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.670809 16913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:48.671000 16913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "994ef811a0e34551a75344f6a17f7197"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "46d9a61f38d5acdb5a29084f6e23c889"
server_key_iv: "f7386e89bf501501f328e4e10b6640a6"
server_key_version: "encryptionkey@0"
I20260811 09:01:48.671212 16913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:48.714571 16913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:48.719568 16913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:48.719862 16913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:48.720968 16913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:48.722637 16913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:48.722728 16913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.722822 16913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:48.722869 16913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:48.760205 16913 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:36897
I20260811 09:01:48.760249 17056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:36897 every 8 connection(s)
I20260811 09:01:48.761835 16913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:48.765180 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 16913
I20260811 09:01:48.765415 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:01:48.765808 9377 external_mini_cluster.cc:1428] Setting key 6cf38c3512ff86f1700322654409e2a3
Aug 11 09:01:48 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438908, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:48.786039 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:48.764933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60059 (local address 127.9.40.126:45611)
0811 09:01:48.765503 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:48.765510 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:48.766797 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 09:01:48.769466 (+ 2669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:48.769472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:48.769589 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:48.769720 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:48.771504 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.772324 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.774184 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.774624 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.774774 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:48.779027 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:48.779033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:48.779037 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:48.779086 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:48.782697 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:48.783622 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:48.783627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:48.783630 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:48.783774 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:48.784191 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:48.784195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:48.784197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:48.784489 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:48.784627 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:48.785509 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 09:01:48.785733 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":270,"threads_started":1}
I20260811 09:01:48.787212 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:48.765634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:60059)
0811 09:01:48.766589 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:48.766643 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:48.769114 (+ 2471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:48.770012 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:48.770022 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:48.770633 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:48.771294 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:48.771315 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.772541 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.772555 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:48.774005 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:48.774020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:48.774887 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:48.774894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:48.775175 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:48.776002 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:48.776029 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:48.778715 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:48.782879 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:48.782889 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:48.782908 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:48.783462 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:48.783873 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:48.783877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:48.783879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:48.784089 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:48.784637 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:48.784644 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:48.785096 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 09:01:48.785412 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 09:01:48.785907 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":359,"threads_started":1}
I20260811 09:01:48.788890 17057 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:48.789268 17057 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:48.790119 17057 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:48.791960 16536 ts_manager.cc:194] Registered new tserver with Master: 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:48.792928 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60059
I20260811 09:01:48.795233 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.125.keytab.
Entry for principal oryx/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.125.keytab.
I20260811 09:01:48.858929 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:34221
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.125.keytab
--principal=oryx/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45611,127.9.40.125:34221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:01:49.195137 16907 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
W20260811 09:01:49.253288 17064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:49.253643 17064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:49.253803 17064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:49.262514 17064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:49.262656 17064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:49.262754 17064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:49.262858 17064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:49.262935 17064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:49.274294 17064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45611,127.9.40.125:34221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.125.keytab
--principal=oryx/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:34221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
W20260811 09:01:49.274765 17064 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.40.126:45611,127.9.40.125:34221'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 09:01:49.276049 17064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:49.277767 17064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:49.291554 17070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:49.291530 17072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:49.291541 17069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:49.292460 17064 server_base.cc:1034] running on GCE node
I20260811 09:01:49.293455 17064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:49.295222 17064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:49.296538 17064 hybrid_clock.cc:648] HybridClock initialized: now 1786438909296461 us; error 62 us; skew 500 ppm
I20260811 09:01:49.297011 17064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438909, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:49.300836 17064 init.cc:377] Logged in from keytab as oryx/127.9.40.125@KRBTEST.COM (short username oryx)
I20260811 09:01:49.302686 17064 webserver.cc:460] Webserver started at http://127.9.40.125:33189/ using document root <none> and password file <none>
I20260811 09:01:49.303555 17064 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:49.303709 17064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:49.304095 17064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:49.307727 17064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/instance:
uuid: "9c3335d4282f461481d2a2fb9631abc4"
format_stamp: "Formatted at 2026-08-11 09:01:49 on dist-test-slave-cbsf"
server_key: "72d618fe8e64cc4e34c0c6ac46009902"
server_key_iv: "9502ff78d46b458e13600a565b80accc"
server_key_version: "encryptionkey@0"
I20260811 09:01:49.308888 17064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal/instance:
uuid: "9c3335d4282f461481d2a2fb9631abc4"
format_stamp: "Formatted at 2026-08-11 09:01:49 on dist-test-slave-cbsf"
server_key: "72d618fe8e64cc4e34c0c6ac46009902"
server_key_iv: "9502ff78d46b458e13600a565b80accc"
server_key_version: "encryptionkey@0"
I20260811 09:01:49.315420 17064 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:01:49.319871 17079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:49.321683 17064 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:01:49.321929 17064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "9c3335d4282f461481d2a2fb9631abc4"
format_stamp: "Formatted at 2026-08-11 09:01:49 on dist-test-slave-cbsf"
server_key: "72d618fe8e64cc4e34c0c6ac46009902"
server_key_iv: "9502ff78d46b458e13600a565b80accc"
server_key_version: "encryptionkey@0"
I20260811 09:01:49.322211 17064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:49.335886 17064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:49.341212 17064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:49.341534 17064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:49.362291 17064 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:34221
I20260811 09:01:49.362309 17143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:34221 every 8 connection(s)
I20260811 09:01:49.363957 17064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
I20260811 09:01:49.368407 17144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:49.369709 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 17064
I20260811 09:01:49.380035 17144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438909, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:49.399878 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.372420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36026 (local address 127.9.40.125:34221)
0811 09:01:49.373414 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.373432 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.373479 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:01:49.375954 (+ 2475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.375990 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.376602 (+ 612us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.377106 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.379159 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.380860 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.382819 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.384083 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.384346 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.390086 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.390101 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.390122 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.390185 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.394130 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.395184 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.395194 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.395204 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.395433 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.395848 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.395853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.395856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.396801 (+ 945us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.397159 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.397736 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.398289 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":465,"threads_started":1}
I20260811 09:01:49.402801 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.383721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.9.40.126:45611)
0811 09:01:49.384280 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.384289 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.385025 (+ 736us) server_negotiation.cc:407] Connection header received
0811 09:01:49.385325 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.385331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.385465 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.385590 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.387024 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.388280 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.390117 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.390598 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.390725 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.395276 (+ 4551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.395284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.395301 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.395372 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.399266 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.400203 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.400211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.400215 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.400398 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.400931 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.400936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.400940 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.401448 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.401631 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.402264 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.402491 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":288,"threads_started":1}
I20260811 09:01:49.402987 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.383694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:39756)
0811 09:01:49.384835 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.384901 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.385173 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.385671 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.385677 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.386549 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.386833 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.386843 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.388458 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.388465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.389928 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.389941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.390819 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.390825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.390999 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.391668 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.391692 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.394983 (+ 3291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.399440 (+ 4457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.399452 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.399458 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.399990 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.400572 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.400576 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.400578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.400801 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.401630 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.401640 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.401869 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.402122 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.402518 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":578,"threads_started":1}
I20260811 09:01:49.405346 17144 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438909, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:49.419198 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.406320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36028 (local address 127.9.40.125:34221)
0811 09:01:49.406645 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.406652 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.406759 (+ 107us) server_negotiation.cc:407] Connection header received
0811 09:01:49.406953 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.406959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.407024 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.407128 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.408037 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.408916 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.410112 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.410538 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.410682 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.413923 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.413929 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.413934 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.413967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.416456 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.417245 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.417252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.417255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.417384 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.417760 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.417768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.417771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.418089 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.418242 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.418582 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.418806 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 09:01:49.419361 17150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.406320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36028)
0811 09:01:49.406656 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.406677 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.406837 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.407151 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.407156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.407688 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.407839 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.407848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.409059 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.409064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.409956 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.409969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.410706 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.410712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.410886 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.411467 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.411486 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.413715 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.416626 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.416633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.416638 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.417104 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.417512 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.417518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.417522 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.417658 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.418200 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.418204 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.418365 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.418507 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.418986 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":153}
I20260811 09:01:49.424897 17144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Bootstrap starting.
I20260811 09:01:49.428683 17144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:49.430003 17144 log.cc:824] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:49.433522 17144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: No bootstrap required, opened a new log
I20260811 09:01:49.441366 17144 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } }
I20260811 09:01:49.441782 17144 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:49.441864 17144 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c3335d4282f461481d2a2fb9631abc4, State: Initialized, Role: FOLLOWER
I20260811 09:01:49.442469 17144 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } }
I20260811 09:01:49.443436 17152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } } }
I20260811 09:01:49.443799 17152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:49.444649 17144 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:01:49.455626 17163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:01:49.455858 17163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:01:49.458963 17064 master_runner.cc:428] Detected that this master 9c3335d4282f461481d2a2fb9631abc4 is joining an existing cluster
I20260811 09:01:49.459100 17064 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.40.125:34221
WARNING: no policy specified for oryx/127.9.40.124@KRBTEST.COM; defaulting to no policy
I20260811 09:01:49.497056 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.469209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.9.40.126:45611)
0811 09:01:49.469464 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.469471 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.477509 (+ 8038us) server_negotiation.cc:407] Connection header received
0811 09:01:49.477971 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.477979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.478096 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.478180 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.479994 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.481176 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.483494 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.484238 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.484358 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.488987 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.488996 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.489001 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.489055 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.493341 (+ 4286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.494386 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.494396 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.494402 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.494602 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.495042 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.495049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.495052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.495383 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.495659 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.496295 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.496625 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:01:49.497394 17173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.469495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:39772)
0811 09:01:49.477315 (+ 7820us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.477345 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.477728 (+ 383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.478448 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.478454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.479264 (+ 810us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.479753 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.479770 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.481400 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.481406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.483246 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.483272 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.484775 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.484785 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.485025 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.487015 (+ 1990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.487046 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.488658 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.493580 (+ 4922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.493586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.493591 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.494177 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.494745 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.494749 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.494751 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.494910 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.495565 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.495570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.495848 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.496139 (+ 291us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.496562 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7607,"spinlock_wait_cycles":1280,"thread_start_us":7407,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.124@KRBTEST.COM" created.
I20260811 09:01:49.503857 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.470314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36036 (local address 127.9.40.125:34221)
0811 09:01:49.470554 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.470561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.486168 (+ 15607us) server_negotiation.cc:407] Connection header received
0811 09:01:49.486469 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.486476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.486563 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.486682 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.488447 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.489815 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.492176 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.492983 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.493104 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.495815 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.495821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.495827 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.495859 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.500088 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.501289 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.501295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.501299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.501522 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.501997 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.502002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.502007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.502425 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.502582 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.503151 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.503435 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:01:49.505282 17176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.469747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36036)
0811 09:01:49.486052 (+ 16305us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.486077 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.486328 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.486739 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.486747 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.487892 (+ 1145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.488256 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.488268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.490074 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.490084 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.491971 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.491994 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.493476 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.493486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.493681 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.494407 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.494431 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.495620 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.500279 (+ 4659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.500286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.500291 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.501118 (+ 827us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.501681 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.501685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.501689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.501870 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.502676 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.502681 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.502844 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.504350 (+ 1506us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.504729 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16079,"spinlock_wait_cycles":45440,"thread_start_us":15827,"threads_started":1}
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:49.535265 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.506071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36048 (local address 127.9.40.125:34221)
0811 09:01:49.506260 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.506267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.513715 (+ 7448us) server_negotiation.cc:407] Connection header received
0811 09:01:49.514406 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.514428 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.514563 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.514702 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.516491 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.517865 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.520267 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.521273 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.521463 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.526815 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.526824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.526830 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.526881 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.531299 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.532551 (+ 1252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.532559 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.532564 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.532811 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.533374 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.533379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.533381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.533806 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.533975 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.534530 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.534810 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:01:49.535418 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.505699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36048)
0811 09:01:49.513510 (+ 7811us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.513572 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.514071 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.514801 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.514811 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.515870 (+ 1059us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.516231 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.516250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.518075 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.518082 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.520019 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.520050 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.521536 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.521544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.521815 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.522718 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.522757 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.526456 (+ 3699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.531689 (+ 5233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.531711 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.531720 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.532372 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.532989 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.532995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.532999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.533229 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.533970 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.533980 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.534208 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.534418 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.534929 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7436,"thread_start_us":7221,"threads_started":1}
I20260811 09:01:49.538098 16536 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.40.125:34221: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4"
member_type: NON_VOTER
last_known_addr {
host: "127.9.40.125"
port: 34221
}
attrs {
promote: true
}
}
dest_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.124.keytab.
Entry for principal oryx/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.124.keytab.
I20260811 09:01:49.539935 16536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } }
I20260811 09:01:49.541070 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:41981
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.124.keytab
--principal=oryx/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:01:49.557960 17114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:49.557863 17181 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } }
I20260811 09:01:49.562458 17114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Refusing update from remote peer 38f97f9a1bfa447ea1f8a39f791b7ee2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 09:01:49.563694 17181 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.40.125:34221
I20260811 09:01:49.565331 17064 master.cc:537] Master@127.9.40.125:34221 shutting down...
I20260811 09:01:49.566964 17181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:49.566845 17182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } }
I20260811 09:01:49.567492 17182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
W20260811 09:01:49.572932 16521 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 09:01:49.584884 16521 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 -> Peer 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125:34221): Couldn't send request to peer 9c3335d4282f461481d2a2fb9631abc4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 09:01:49.587888 17064 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:49.588394 17064 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:49.588616 17064 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: stopping tablet replica
I20260811 09:01:49.596829 17064 master.cc:552] Master@127.9.40.125:34221 shutdown complete.
I20260811 09:01:49.597012 17064 master_runner.cc:305] Clearing existing system tablet
I20260811 09:01:49.597610 17064 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:49.597731 17064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:49.601202 17064 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260811 09:01:49.603190 17188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:49.603606 17064 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:49.603884 17064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 09:01:49.603983 17064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "9c3335d4282f461481d2a2fb9631abc4"
format_stamp: "Formatted at 2026-08-11 09:01:49 on dist-test-slave-cbsf"
server_key: "72d618fe8e64cc4e34c0c6ac46009902"
server_key_iv: "9502ff78d46b458e13600a565b80accc"
server_key_version: "encryptionkey@0"
I20260811 09:01:49.609885 17064 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:49.617801 17064 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:01:49.618204 17064 log.cc:1196] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 09:01:49.618773 17064 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Deleting consensus metadata
I20260811 09:01:49.619086 17064 master_runner.cc:315] Copying system tablet from 127.9.40.126:45611
I20260811 09:01:49.622432 17064 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.126:45611
I20260811 09:01:49.642827 17196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.623363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:39788)
0811 09:01:49.624044 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:49.624073 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:49.624255 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:49.624710 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:49.624715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:49.625331 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:49.625690 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.625712 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.627511 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.627520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.629545 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:49.629597 (+ 52us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.630900 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.630911 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:49.631189 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.632069 (+ 880us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:49.632096 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:49.633371 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:49.638975 (+ 5604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.638981 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.638990 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.639608 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.640213 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:49.640219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:49.640222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:49.640425 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:49.641521 (+ 1096us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:49.641526 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:49.641776 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 09:01:49.641944 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 09:01:49.642330 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":391,"threads_started":1}
I20260811 09:01:49.644196 17149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:49.623569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.9.40.126:45611)
0811 09:01:49.623774 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:49.623782 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:49.624193 (+ 411us) server_negotiation.cc:407] Connection header received
0811 09:01:49.624411 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:49.624418 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:49.624509 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:49.624625 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:49.625914 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.627239 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.629849 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:49.630647 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:49.630821 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:49.633672 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:49.633681 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:49.633686 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:49.633736 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:49.638701 (+ 4965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.639811 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.639818 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.639821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.640044 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:49.640592 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:49.640597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:49.640601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:49.641219 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:49.641406 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:49.643479 (+ 2073us) server_negotiation.cc:299] Negotiation successful
0811 09:01:49.643791 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:01:49.646862 16558 tablet_copy_service.cc:140] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9c3335d4282f461481d2a2fb9631abc4 ({username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:39788)
I20260811 09:01:49.647342 16558 tablet_copy_service.cc:161] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9c3335d4282f461481d2a2fb9631abc4 at {username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:39788: session id = 9c3335d4282f461481d2a2fb9631abc4-00000000000000000000000000000000
I20260811 09:01:49.652539 16558 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:01:49.655977 17064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:01:49.664146 17064 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: tablet copy: Starting download of 0 data blocks...
I20260811 09:01:49.664641 17064 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: tablet copy: Starting download of 1 WAL segments...
I20260811 09:01:49.668936 17064 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:01:49.675117 16558 tablet_copy_service.cc:342] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Request end of tablet copy session 9c3335d4282f461481d2a2fb9631abc4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:39788
I20260811 09:01:49.675685 16558 tablet_copy_service.cc:433] P 38f97f9a1bfa447ea1f8a39f791b7ee2: ending tablet copy session 9c3335d4282f461481d2a2fb9631abc4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9c3335d4282f461481d2a2fb9631abc4
I20260811 09:01:49.692772 17064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:49.696231 17198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:49.698333 17199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:49.710163 17201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:49.710704 17064 server_base.cc:1034] running on GCE node
I20260811 09:01:49.711100 17064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:49.711876 17064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:49.713191 17064 hybrid_clock.cc:648] HybridClock initialized: now 1786438909713123 us; error 42 us; skew 500 ppm
I20260811 09:01:49.713474 17064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:49 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438909, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:49.716051 17064 init.cc:377] Logged in from keytab as oryx/127.9.40.125@KRBTEST.COM (short username oryx)
I20260811 09:01:49.722358 17064 webserver.cc:460] Webserver started at http://127.9.40.125:35997/ using document root <none> and password file <none>
I20260811 09:01:49.722849 17064 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:49.722946 17064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:49.726440 17064 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 09:01:49.728953 17208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:49.729472 17064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260811 09:01:49.729660 17064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "9c3335d4282f461481d2a2fb9631abc4"
format_stamp: "Formatted at 2026-08-11 09:01:49 on dist-test-slave-cbsf"
server_key: "72d618fe8e64cc4e34c0c6ac46009902"
server_key_iv: "9502ff78d46b458e13600a565b80accc"
server_key_version: "encryptionkey@0"
I20260811 09:01:49.729857 17064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:49.747069 17064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:49.751838 17064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:49.792649 17064 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:34221
I20260811 09:01:49.792680 17263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:34221 every 8 connection(s)
I20260811 09:01:49.794047 17064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
I20260811 09:01:49.796530 17057 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
I20260811 09:01:49.801342 17264 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:01:49.806479 17264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Bootstrap starting.
I20260811 09:01:49.830029 17264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:01:49.831215 17264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: Bootstrap complete.
I20260811 09:01:49.832809 17264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } }
I20260811 09:01:49.833243 17264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9c3335d4282f461481d2a2fb9631abc4, State: Initialized, Role: LEARNER
I20260811 09:01:49.833567 17264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } }
I20260811 09:01:49.849890 17268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } }
I20260811 09:01:49.850378 17268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:01:49.850746 17264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:49.861500 17285 catalog_manager.cc:1259] Loaded cluster ID: 49232210f17d47eb9a822f81a921a51e
I20260811 09:01:49.861654 17285 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: loading cluster ID for follower catalog manager: success
I20260811 09:01:49.865861 17285 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: acquiring CA information for follower catalog manager: success
I20260811 09:01:49.869184 17285 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:01:50.026924 17183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:50.027346 17183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:50.027503 17183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 09:01:50.036726 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.028965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36060)
0811 09:01:50.029224 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.029251 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.029411 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.030270 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.030277 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.030313 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:50.030561 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.030573 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.032141 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.032148 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.034380 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.034408 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.035781 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.035788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.036045 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.036119 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.036228 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.036393 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 09:01:50.036687 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.029198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36060 (local address 127.9.40.125:34221)
0811 09:01:50.029862 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.029866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.029891 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:01:50.029961 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.029965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.030056 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.030174 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:50.030729 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.031953 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.034605 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.035562 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.035681 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.035912 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.036282 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.036412 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":379,"threads_started":1}
W20260811 09:01:50.038033 17183 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:50.038199 17183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260811 09:01:50.038025 17235 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
W20260811 09:01:50.038318 17183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:50.038420 17183 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:50.038514 17183 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:50.038841 17268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f97f9a1bfa447ea1f8a39f791b7ee2. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } }
I20260811 09:01:50.039148 17268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:01:50.049924 17183 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.124.keytab
--principal=oryx/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:41981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:50.051745 17183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:50.053345 17183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:50.068492 17292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:50.068475 17291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:50.068504 17294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:50.069087 17183 server_base.cc:1034] running on GCE node
I20260811 09:01:50.070122 17183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:50.072029 17183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:50.073279 17183 hybrid_clock.cc:648] HybridClock initialized: now 1786438910073211 us; error 64 us; skew 500 ppm
I20260811 09:01:50.073666 17183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438910, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:50.077587 17183 init.cc:377] Logged in from keytab as oryx/127.9.40.124@KRBTEST.COM (short username oryx)
I20260811 09:01:50.079442 17183 webserver.cc:460] Webserver started at http://127.9.40.124:33777/ using document root <none> and password file <none>
I20260811 09:01:50.080353 17183 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:50.080504 17183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:50.080936 17183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:50.084632 17183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/instance:
uuid: "086f89fe0d4e44a1aa325df64a13e68e"
format_stamp: "Formatted at 2026-08-11 09:01:50 on dist-test-slave-cbsf"
server_key: "d9ec2cae2c28264e26ec10b3f07f8f95"
server_key_iv: "fa6be621a5d8aff4a992cca6abeb1524"
server_key_version: "encryptionkey@0"
I20260811 09:01:50.085784 17183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal/instance:
uuid: "086f89fe0d4e44a1aa325df64a13e68e"
format_stamp: "Formatted at 2026-08-11 09:01:50 on dist-test-slave-cbsf"
server_key: "d9ec2cae2c28264e26ec10b3f07f8f95"
server_key_iv: "fa6be621a5d8aff4a992cca6abeb1524"
server_key_version: "encryptionkey@0"
I20260811 09:01:50.092315 17183 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:01:50.096495 17306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:50.098474 17183 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:50.098733 17183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "086f89fe0d4e44a1aa325df64a13e68e"
format_stamp: "Formatted at 2026-08-11 09:01:50 on dist-test-slave-cbsf"
server_key: "d9ec2cae2c28264e26ec10b3f07f8f95"
server_key_iv: "fa6be621a5d8aff4a992cca6abeb1524"
server_key_version: "encryptionkey@0"
I20260811 09:01:50.098958 17183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:50.121451 17183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:50.126768 17183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:50.127156 17183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:50.149416 17183 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:41981
I20260811 09:01:50.149427 17363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:41981 every 8 connection(s)
I20260811 09:01:50.151221 17183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
I20260811 09:01:50.156736 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 17183
I20260811 09:01:50.157541 17364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:50.170210 17364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438910, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:50.188122 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.159994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56548 (local address 127.9.40.124:41981)
0811 09:01:50.161579 (+ 1585us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.161599 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.162060 (+ 461us) server_negotiation.cc:407] Connection header received
0811 09:01:50.164572 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.164611 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.165324 (+ 713us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.165880 (+ 556us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.167563 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.169668 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.171075 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.172220 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.172425 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.178205 (+ 5780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.178214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.178226 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.178275 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.182340 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.183380 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.183391 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.183413 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.183631 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.184154 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.184160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.184170 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.185038 (+ 868us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.185419 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.186032 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.186558 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1276,"thread_start_us":946,"threads_started":1}
I20260811 09:01:50.192399 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.173622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39798 (local address 127.9.40.126:45611)
0811 09:01:50.175023 (+ 1401us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.175031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.175053 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:01:50.175140 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.175146 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.175239 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.175383 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.176816 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.177637 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.179330 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.179833 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.179989 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.183906 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.183913 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.183916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.183957 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.187207 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.188362 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.188372 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.188378 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.189862 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.190435 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.190439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.190441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.190804 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.190943 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.191648 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.191993 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1257,"thread_start_us":1129,"threads_started":1}
I20260811 09:01:50.192416 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.173466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:39798)
0811 09:01:50.174414 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.174476 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.174694 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.175434 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.175439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.176187 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.176615 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.176630 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.177798 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.177804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.179149 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.179168 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.180036 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.180044 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.180360 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.181230 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.181250 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.183648 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.187388 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.187401 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.187410 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.188163 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.190030 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.190036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.190040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.190297 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.190986 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.190994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.191223 (+ 229us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.191526 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.191983 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":368,"threads_started":1}
I20260811 09:01:50.193966 17364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438910, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:50.209676 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.194789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36068)
0811 09:01:50.195016 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.195038 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.195152 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.195495 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.195498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.195911 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.196083 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.196092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.197328 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.197332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.198521 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.198539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.199379 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.199384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.199619 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.200253 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.200271 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.202883 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.206472 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.206478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.206493 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.207300 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.207707 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.207715 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.207721 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.207918 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.208543 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.208549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.208771 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.208918 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.209307 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:01:50.209676 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.194942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.9.40.125:34221)
0811 09:01:50.195146 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.195151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.195170 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:50.195259 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.195263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.195329 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.195419 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.196245 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.197092 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.198708 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.199165 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.199274 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.203214 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.203220 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.203224 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.203265 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.206271 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.207436 (+ 1165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.207441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.207443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.207584 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.208063 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.208067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.208070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.208411 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.208582 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.209044 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.209307 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:01:50.211131 17364 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438910, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260811 09:01:50.229913 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.212542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:56560)
0811 09:01:50.212793 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.212813 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.212933 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.213557 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.213561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.213948 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.214099 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.214108 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.215833 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.215837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.216768 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.216787 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.218068 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.218073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.218243 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.218932 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.218953 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.221732 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.226724 (+ 4992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.226730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.226733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.227225 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.227752 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.227756 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.227759 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.227955 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.228882 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.228886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.229055 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.229204 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.229440 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:01:50.230321 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.212804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56560 (local address 127.9.40.124:41981)
0811 09:01:50.213145 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.213160 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.213191 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:01:50.213277 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.213284 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.213381 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.213527 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.214262 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.215665 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.216956 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.217866 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.218051 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.222025 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.222035 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.222043 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.222094 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.226478 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.227384 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.227403 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.227410 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.227591 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.228099 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.228106 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.228110 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.228686 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.228918 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.229380 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.229743 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":83}
I20260811 09:01:50.238131 17364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Bootstrap starting.
I20260811 09:01:50.241698 17364 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:50.243834 17364 log.cc:824] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:50.247107 17364 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: No bootstrap required, opened a new log
I20260811 09:01:50.258876 17364 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } }
I20260811 09:01:50.259464 17364 raft_consensus.cc:374] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:50.259698 17364 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086f89fe0d4e44a1aa325df64a13e68e, State: Initialized, Role: FOLLOWER
I20260811 09:01:50.260623 17364 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } }
I20260811 09:01:50.262451 17373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } } }
I20260811 09:01:50.262961 17373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:50.263877 17364 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:01:50.280154 17384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:01:50.280387 17384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:01:50.286959 17183 master_runner.cc:428] Detected that this master 086f89fe0d4e44a1aa325df64a13e68e is joining an existing cluster
I20260811 09:01:50.287251 17183 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.40.124:41981
I20260811 09:01:50.297210 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16602
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.65@KRBTEST.COM
I20260811 09:01:50.327528 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.294431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39812 (local address 127.9.40.126:45611)
0811 09:01:50.294716 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.294723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.300756 (+ 6033us) server_negotiation.cc:407] Connection header received
0811 09:01:50.301348 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.301354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.301459 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.301556 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.305318 (+ 3762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.306917 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.310574 (+ 3657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.311593 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.311765 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.316790 (+ 5025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.316800 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.316806 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.316875 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.322636 (+ 5761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.323890 (+ 1254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.323897 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.323900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.324182 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.324802 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.324808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.324812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.325441 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.325669 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.326644 (+ 975us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.326977 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 09:01:50.327943 17391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.295652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:39812)
0811 09:01:50.300593 (+ 4941us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.300619 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.300932 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.301913 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.301919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.303691 (+ 1772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.305033 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.305060 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.307196 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.307201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.310405 (+ 3204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.310422 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.313003 (+ 2581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.313011 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.313238 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.314263 (+ 1025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.314288 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.316127 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.322796 (+ 6669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.322804 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.322811 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.323714 (+ 903us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.324322 (+ 608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.324328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.324333 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.324629 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.325572 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.325577 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.326279 (+ 702us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.326522 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.327067 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4680,"thread_start_us":4290,"threads_started":1}
I20260811 09:01:50.339542 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.294694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36076 (local address 127.9.40.125:34221)
0811 09:01:50.294876 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.294880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.297574 (+ 2694us) server_negotiation.cc:407] Connection header received
0811 09:01:50.297878 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.297884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.297976 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.298085 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.308726 (+ 10641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.310202 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.315139 (+ 4937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.316207 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.316320 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.330250 (+ 13930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.330260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.330265 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.330322 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.335214 (+ 4892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.336171 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.336193 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.336198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.336391 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.337096 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.337102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.337107 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.337646 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.337791 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.338728 (+ 937us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.339117 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:01:50.340085 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.300967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56572 (local address 127.9.40.124:41981)
0811 09:01:50.301193 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.301198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.305288 (+ 4090us) server_negotiation.cc:407] Connection header received
0811 09:01:50.305535 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.305540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.305613 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.305683 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.311078 (+ 5395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.312646 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.319346 (+ 6700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.320616 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.320782 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.329134 (+ 8352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.329142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.329146 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.329184 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.335830 (+ 6646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.336934 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.336940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.336943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.337175 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.337659 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.337664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.337667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.338391 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.338568 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.339151 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.339405 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:01:50.343586 17390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.295577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:36076)
0811 09:01:50.297409 (+ 1832us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.297432 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.297726 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.298120 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.298125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.299374 (+ 1249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.308457 (+ 9083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.308474 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.312149 (+ 3675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.312154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.314819 (+ 2665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.314848 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.323193 (+ 8345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.323206 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.323530 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.328000 (+ 4470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.328165 (+ 165us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.329901 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.335414 (+ 5513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.335420 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.335425 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.335988 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.336575 (+ 587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.336580 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.336584 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.336934 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.337883 (+ 949us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.337890 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.338294 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.340147 (+ 1853us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.340839 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1645,"mutex_wait_us":1,"spinlock_wait_cycles":33536,"thread_start_us":1393,"threads_started":1}
I20260811 09:01:50.344290 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.301426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:56572)
0811 09:01:50.305178 (+ 3752us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.305203 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.305407 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.305915 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.305919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.307329 (+ 1410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.310905 (+ 3576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.310926 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.316413 (+ 5487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.316420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.319116 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.319158 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.321246 (+ 2088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.321255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.321487 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.322502 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.322535 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.328877 (+ 6342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.335997 (+ 7120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.336002 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.336007 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.336797 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.337334 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.337338 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.337341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.337537 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.338640 (+ 1103us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.338645 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.338848 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.339802 (+ 954us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.342340 (+ 2538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3465,"thread_start_us":2915,"threads_started":1}
I20260811 09:01:50.349833 17392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.299208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.9.40.65:42497)
0811 09:01:50.300435 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.300444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.300491 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:01:50.300624 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.300635 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.300991 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.303948 (+ 2957us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.304481 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.306801 (+ 2320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.310009 (+ 3208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.311400 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.320330 (+ 8930us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.320904 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.320924 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.320947 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.321006 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.327312 (+ 6306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.334805 (+ 7493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.334830 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.334834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.335108 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.345389 (+ 10281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.345395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.345399 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.346043 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.346485 (+ 442us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.348340 (+ 1855us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.348873 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":1851,"threads_started":1}
I20260811 09:01:50.352365 16745 generic_service.cc:196] Checking for leaks (request via RPC)
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260811 09:01:50.364533 17367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.338436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36998 (local address 127.9.40.124:41981)
0811 09:01:50.340334 (+ 1898us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.340339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.340357 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:01:50.340466 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.340472 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.340613 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.341821 (+ 1208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.342329 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.344457 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.346885 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.348146 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.354346 (+ 6200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.354683 (+ 337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.354690 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.354696 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.354735 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.360802 (+ 6067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.361754 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.361760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.361768 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.361949 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.362439 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.362445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.362448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.362869 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.363053 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.363635 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.363923 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1791,"thread_start_us":13494,"threads_started":1}
I20260811 09:01:50.364562 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.337341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:36998)
0811 09:01:50.337986 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.338021 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.338147 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.340862 (+ 2715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.340870 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.341435 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.341681 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.341693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.344676 (+ 2983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.344684 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.346591 (+ 1907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.346615 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.348384 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.348392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.348732 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.349683 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.349712 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.354047 (+ 4335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.361036 (+ 6989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.361048 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.361051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.361586 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.362114 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.362120 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.362124 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.362300 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.363014 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.363020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.363276 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.363447 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.363914 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465}
I20260811 09:01:50.366488 16536 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.40.124:41981: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e"
member_type: NON_VOTER
last_known_addr {
host: "127.9.40.124"
port: 41981
}
attrs {
promote: true
}
}
dest_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2"
cas_config_opid_index: 5
I20260811 09:01:50.367687 16536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } }
I20260811 09:01:50.369961 17182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.377564 17182 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.40.124:41981
I20260811 09:01:50.378623 17183 master.cc:537] Master@127.9.40.124:41981 shutting down...
W20260811 09:01:50.380169 17312 tablet.cc:2369] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:50.402963 16518 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 -> Peer 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124:41981): Couldn't send request to peer 086f89fe0d4e44a1aa325df64a13e68e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 09:01:50.404098 17235 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Refusing update from remote peer 38f97f9a1bfa447ea1f8a39f791b7ee2: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 09:01:50.408310 17398 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260811 09:01:50.409729 17183 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:50.410255 17183 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:50.410528 17183 tablet_replica.cc:330] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: stopping tablet replica
I20260811 09:01:50.412451 17397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.413141 17397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:50.425714 17235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.430617 17268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: true } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.431156 17268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:01:50.432788 17183 master.cc:552] Master@127.9.40.124:41981 shutdown complete.
I20260811 09:01:50.432835 17398 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: attempting to promote NON_VOTER 9c3335d4282f461481d2a2fb9631abc4 to VOTER
I20260811 09:01:50.432951 17183 master_runner.cc:305] Clearing existing system tablet
I20260811 09:01:50.433575 17183 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:50.433844 17183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:50.434193 17398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } }
I20260811 09:01:50.438920 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.437871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:37014)
0811 09:01:50.438399 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.438577 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:41981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350}
I20260811 09:01:50.441538 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.440594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:37030)
0811 09:01:50.440923 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.441066 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:41981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
W20260811 09:01:50.441897 16518 consensus_peers.cc:487] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 -> Peer 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124:41981): Couldn't send request to peer 086f89fe0d4e44a1aa325df64a13e68e. Status: Network error: Client connection negotiation failed: client connection to 127.9.40.124:41981: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:01:50.442691 17235 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 LEARNER]: Refusing update from remote peer 38f97f9a1bfa447ea1f8a39f791b7ee2: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 09:01:50.443814 17182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 09:01:50.447849 17397 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.449074 17235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.454569 17182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.455102 17182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:50.454949 17268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.455312 17268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:50.455564 17182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.455909 17182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:50.456393 17183 fs_manager.cc:709] Time spent opening directory manager: real 0.021s user 0.001s sys 0.013s
I20260811 09:01:50.459022 17405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:50.459641 17183 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:50.460055 17183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:01:50.460258 17183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "086f89fe0d4e44a1aa325df64a13e68e"
format_stamp: "Formatted at 2026-08-11 09:01:50 on dist-test-slave-cbsf"
server_key: "d9ec2cae2c28264e26ec10b3f07f8f95"
server_key_iv: "fa6be621a5d8aff4a992cca6abeb1524"
server_key_version: "encryptionkey@0"
I20260811 09:01:50.465430 17183 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:50.473021 17183 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:01:50.473366 17183 log.cc:1196] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 09:01:50.474087 17183 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Deleting consensus metadata
I20260811 09:01:50.474426 17183 master_runner.cc:315] Copying system tablet from 127.9.40.126:45611
I20260811 09:01:50.482443 17183 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.126:45611
I20260811 09:01:50.503358 17370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.483534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.9.40.126:45611)
0811 09:01:50.483901 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.483911 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.485866 (+ 1955us) server_negotiation.cc:407] Connection header received
0811 09:01:50.486175 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.486186 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.486261 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.486376 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:50.487862 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.489335 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.491504 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.492364 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.492553 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.495528 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:50.495535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:50.495539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:50.495580 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:50.499629 (+ 4049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.500530 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.500543 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.500546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.500867 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:50.501359 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:50.501365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:50.501369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:50.501759 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:50.501951 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.502537 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.502809 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213}
I20260811 09:01:50.503577 17412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.483592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34534)
0811 09:01:50.485732 (+ 2140us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.485753 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.485992 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.486421 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.486426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.487300 (+ 874us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:50.487676 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.487689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.489585 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.489596 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.491285 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.491329 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.492581 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.492588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.492992 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.493973 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:50.493992 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:50.495097 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:50.499856 (+ 4759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.499863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.499868 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.500368 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.501010 (+ 642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:50.501016 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:50.501020 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:50.501221 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:50.501927 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:50.501941 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:50.502235 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.502422 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.502822 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1860,"thread_start_us":1646,"threads_started":1}
I20260811 09:01:50.504618 16558 tablet_copy_service.cc:140] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 086f89fe0d4e44a1aa325df64a13e68e ({username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:34534)
I20260811 09:01:50.504907 16558 tablet_copy_service.cc:161] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 086f89fe0d4e44a1aa325df64a13e68e at {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:34534: session id = 086f89fe0d4e44a1aa325df64a13e68e-00000000000000000000000000000000
I20260811 09:01:50.508486 16558 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:01:50.510839 17183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:01:50.518220 17183 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: tablet copy: Starting download of 0 data blocks...
I20260811 09:01:50.518728 17183 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: tablet copy: Starting download of 1 WAL segments...
I20260811 09:01:50.522078 17183 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:01:50.526893 16558 tablet_copy_service.cc:342] P 38f97f9a1bfa447ea1f8a39f791b7ee2: Request end of tablet copy session 086f89fe0d4e44a1aa325df64a13e68e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:34534
I20260811 09:01:50.527117 16558 tablet_copy_service.cc:433] P 38f97f9a1bfa447ea1f8a39f791b7ee2: ending tablet copy session 086f89fe0d4e44a1aa325df64a13e68e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 086f89fe0d4e44a1aa325df64a13e68e
I20260811 09:01:50.539428 17183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:50.544533 17414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:50.549270 17416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:50.551532 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16602
W20260811 09:01:50.554481 17419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:50.567265 17183 server_base.cc:1034] running on GCE node
I20260811 09:01:50.567742 17183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:50.568583 17183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:50.569839 17183 hybrid_clock.cc:648] HybridClock initialized: now 1786438910569782 us; error 51 us; skew 500 ppm
I20260811 09:01:50.570202 17183 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:50 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438910, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:50.572971 17183 init.cc:377] Logged in from keytab as oryx/127.9.40.124@KRBTEST.COM (short username oryx)
I20260811 09:01:50.581794 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:42497
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:45611
--webserver_port=38649
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:01:50.583364 17183 webserver.cc:460] Webserver started at http://127.9.40.124:42109/ using document root <none> and password file <none>
I20260811 09:01:50.583756 17183 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:50.583894 17183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:50.587015 17183 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:01:50.589257 17427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:50.589792 17183 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 09:01:50.589956 17183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "086f89fe0d4e44a1aa325df64a13e68e"
format_stamp: "Formatted at 2026-08-11 09:01:50 on dist-test-slave-cbsf"
server_key: "d9ec2cae2c28264e26ec10b3f07f8f95"
server_key_iv: "fa6be621a5d8aff4a992cca6abeb1524"
server_key_version: "encryptionkey@0"
I20260811 09:01:50.590186 17183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:50.623620 17183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:50.630193 17183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:50.697710 17183 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:41981
I20260811 09:01:50.697793 17486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:41981 every 8 connection(s)
I20260811 09:01:50.699049 17183 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
I20260811 09:01:50.705257 17487 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:01:50.710021 17487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Bootstrap starting.
I20260811 09:01:50.733057 17487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:01:50.733729 17487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Bootstrap complete.
I20260811 09:01:50.735036 17487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } }
I20260811 09:01:50.735414 17487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 086f89fe0d4e44a1aa325df64a13e68e, State: Initialized, Role: LEARNER
I20260811 09:01:50.735713 17487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } }
I20260811 09:01:50.736755 17490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.737102 17487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:50.737277 17490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: LEARNER
I20260811 09:01:50.749462 17502 catalog_manager.cc:1259] Loaded cluster ID: 49232210f17d47eb9a822f81a921a51e
I20260811 09:01:50.749606 17502 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: loading cluster ID for follower catalog manager: success
I20260811 09:01:50.753733 17502 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: acquiring CA information for follower catalog manager: success
I20260811 09:01:50.756744 17502 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:01:50.981410 17503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.973536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:37040)
0811 09:01:50.974120 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:50.974153 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:50.974322 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:50.974982 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:50.974987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:50.975022 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:50.975282 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.975292 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.977316 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.977321 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.978588 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:50.978604 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.980588 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.980593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:50.980890 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.980923 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:01:50.981026 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 09:01:50.981154 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
I20260811 09:01:50.981617 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:50.973741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.9.40.124:41981)
0811 09:01:50.974483 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:50.974491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:50.974520 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:01:50.974606 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:50.974614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:50.974723 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:50.974911 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:50.975441 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.977123 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.978794 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:50.980314 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:50.980494 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:50.980847 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:50.981085 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 09:01:50.981260 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":400,"threads_started":1}
I20260811 09:01:50.982764 17456 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 09:01:50.983549 17490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38f97f9a1bfa447ea1f8a39f791b7ee2. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:50.983918 17490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: LEARNER
W20260811 09:01:51.070910 17431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:51.071355 17431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:51.071609 17431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:51.082473 17431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:51.082748 17431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:51.082971 17431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:51.096997 17431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:42497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=38649
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:51.098858 17431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:51.100800 17431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:51.119334 17509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:51.119259 17510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:51.119315 17512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:51.120836 17431 server_base.cc:1034] running on GCE node
I20260811 09:01:51.121649 17431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:51.122895 17431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:51.124202 17431 hybrid_clock.cc:648] HybridClock initialized: now 1786438911124110 us; error 58 us; skew 500 ppm
I20260811 09:01:51.124790 17431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:51.129701 17431 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260811 09:01:51.132220 17431 webserver.cc:460] Webserver started at http://127.9.40.65:38649/ using document root <none> and password file <none>
I20260811 09:01:51.133311 17431 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:51.133512 17431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:51.140663 17431 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:01:51.144948 17522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:51.146716 17431 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 09:01:51.146987 17431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "bcf48d96894746e08e1ca33b78bea683"
format_stamp: "Formatted at 2026-08-11 09:01:47 on dist-test-slave-cbsf"
server_key: "6d536535dc4bc6d049fb65c7956af196"
server_key_iv: "668982991ce68e212c1f842876af6801"
server_key_version: "encryptionkey@0"
I20260811 09:01:51.148437 17431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:51.167783 17431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:51.175264 17431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:51.175814 17431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:51.177316 17431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:51.179761 17431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:51.180001 17431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:51.180254 17431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:51.180469 17431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:51.260810 17431 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42497
I20260811 09:01:51.261125 17650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42497 every 8 connection(s)
I20260811 09:01:51.262923 17431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:51.264251 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 17431
I20260811 09:01:51.264775 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16758
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.66@KRBTEST.COM
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260811 09:01:51.312523 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.266981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36696 (local address 127.9.40.66:46395)
0811 09:01:51.268140 (+ 1159us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:51.268155 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:51.269976 (+ 1821us) server_negotiation.cc:407] Connection header received
0811 09:01:51.270379 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:51.270394 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:51.270813 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:51.270983 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:51.280819 (+ 9836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.283665 (+ 2846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.287173 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.288413 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.288513 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.300398 (+ 11885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:51.300426 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:51.300451 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:51.300532 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:51.307150 (+ 6618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.308654 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.308669 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.308673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.309032 (+ 359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.309969 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.309978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.309984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.310616 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:51.311113 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:51.311471 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 09:01:51.311839 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898,"thread_start_us":616,"threads_started":1}
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:51.314611 16900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:51.320196 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.271786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44873 (local address 127.9.40.126:45611)
0811 09:01:51.275520 (+ 3734us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:51.275527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:51.284413 (+ 8886us) server_negotiation.cc:407] Connection header received
0811 09:01:51.284755 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:51.284763 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:51.284871 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:51.284997 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:51.288838 (+ 3841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.290505 (+ 1667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.296443 (+ 5938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.297580 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.297754 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.304426 (+ 6672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:51.304437 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:51.304442 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:51.304530 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:51.310354 (+ 5824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.315835 (+ 5481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.315843 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.315849 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.316151 (+ 302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.316991 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.316998 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.317001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.317693 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:51.317869 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:51.319103 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0811 09:01:51.319533 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3587,"thread_start_us":3441,"threads_started":1}
I20260811 09:01:51.323545 17658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.271480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:44873)
0811 09:01:51.284247 (+ 12767us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:51.284279 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:51.284556 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:51.285138 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:51.285147 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:51.286734 (+ 1587us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:51.288504 (+ 1770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.288549 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.290775 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.290782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.295506 (+ 4724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.295536 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.298008 (+ 2472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.298022 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.298596 (+ 574us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.299828 (+ 1232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:51.299892 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:51.303993 (+ 4101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:51.313622 (+ 9629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.313635 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.313656 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.314674 (+ 1018us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.316327 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.316335 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.316341 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.316829 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.317844 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:51.317858 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:51.318510 (+ 652us) client_negotiation.cc:769] Sending connection context
0811 09:01:51.320810 (+ 2300us) client_negotiation.cc:240] Negotiation successful
0811 09:01:51.321551 (+ 741us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12481,"mutex_wait_us":444,"thread_start_us":13226,"threads_started":1}
I20260811 09:01:51.323618 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.269743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:49249 (local address 127.9.40.124:41981)
0811 09:01:51.270028 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:51.270037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:51.270909 (+ 872us) server_negotiation.cc:407] Connection header received
0811 09:01:51.276565 (+ 5656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:51.276578 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:51.276851 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:51.276986 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:51.290211 (+ 13225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.291869 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.295524 (+ 3655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.296749 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.296888 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.310458 (+ 13570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:51.310469 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:51.310473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:51.310528 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:51.318037 (+ 7509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.320111 (+ 2074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.320118 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.320122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.320509 (+ 387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.321265 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.321271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.321275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.321821 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:51.321980 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:51.322691 (+ 711us) server_negotiation.cc:299] Negotiation successful
0811 09:01:51.323088 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 09:01:51.327921 17662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.282783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58653 (local address 127.9.40.125:34221)
0811 09:01:51.292043 (+ 9260us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:51.292052 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:51.296626 (+ 4574us) server_negotiation.cc:407] Connection header received
0811 09:01:51.297103 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:51.297110 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:51.297330 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:51.297453 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:51.300408 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.301729 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.304853 (+ 3124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.305799 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.306025 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.314422 (+ 8397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:51.314430 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:51.314435 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:51.314493 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:51.322480 (+ 7987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.324287 (+ 1807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.324307 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.324311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.324549 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:51.325378 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:51.325385 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:51.325389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:51.325971 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:51.326136 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:51.326803 (+ 667us) server_negotiation.cc:299] Negotiation successful
0811 09:01:51.327191 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9047,"thread_start_us":9286,"threads_started":1}
I20260811 09:01:51.330569 17661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.281779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.9.40.65:58653)
0811 09:01:51.296465 (+ 14686us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:51.296491 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:51.296803 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:51.297815 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:51.297820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:51.299164 (+ 1344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:51.300181 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.300204 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.301936 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.301945 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.304261 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.304282 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.306420 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.306431 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.306732 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.308222 (+ 1490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:51.308242 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:51.314013 (+ 5771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:51.322664 (+ 8651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.322670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.322674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.323446 (+ 772us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.324751 (+ 1305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.324770 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.324775 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.325176 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.326151 (+ 975us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:51.326158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:51.326359 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 09:01:51.326548 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 09:01:51.329889 (+ 3341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14454,"mutex_wait_us":2896,"thread_start_us":14028,"threads_started":1}
I20260811 09:01:51.331724 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:51.268732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.9.40.65:49249)
0811 09:01:51.270699 (+ 1967us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:51.270755 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:51.275867 (+ 5112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:51.277995 (+ 2128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:51.278018 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:51.289075 (+ 11057us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:51.289991 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.290011 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.292235 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.292242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.295206 (+ 2964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:51.295234 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:51.300621 (+ 5387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:51.300634 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:51.301077 (+ 443us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:51.304691 (+ 3614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:51.304789 (+ 98us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:51.310073 (+ 5284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:51.318989 (+ 8916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.318995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.318999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.319892 (+ 893us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.320746 (+ 854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:51.320752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:51.320755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:51.321080 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:51.322130 (+ 1050us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:51.322136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:51.322375 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 09:01:51.323847 (+ 1472us) client_negotiation.cc:240] Negotiation successful
0811 09:01:51.331033 (+ 7186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"mutex_wait_us":37,"thread_start_us":1131,"threads_started":1}
I20260811 09:01:51.332327 17653 heartbeater.cc:347] Connected to a master server at 127.9.40.125:34221
I20260811 09:01:51.332944 17653 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:51.333900 17652 heartbeater.cc:347] Connected to a master server at 127.9.40.124:41981
I20260811 09:01:51.334115 17652 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:51.334368 17653 heartbeater.cc:511] Master 127.9.40.125:34221 requested a full tablet report, sending...
I20260811 09:01:51.335239 17652 heartbeater.cc:511] Master 127.9.40.124:41981 requested a full tablet report, sending...
I20260811 09:01:51.335608 17651 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:51.335770 17651 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:51.336467 17651 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:51.337625 17225 ts_manager.cc:194] Registered new tserver with Master: bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:51.338073 17446 ts_manager.cc:194] Registered new tserver with Master: bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:51.339412 16536 ts_manager.cc:194] Re-registered known tserver with Master: bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:51.340673 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:44873
I20260811 09:01:51.393187 17397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: true } } }
I20260811 09:01:51.393805 17397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:51.395764 17665 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: attempting to promote NON_VOTER 086f89fe0d4e44a1aa325df64a13e68e to VOTER
I20260811 09:01:51.396487 17665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } }
I20260811 09:01:51.398869 17235 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Refusing update from remote peer 38f97f9a1bfa447ea1f8a39f791b7ee2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:01:51.399196 17456 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 LEARNER]: Refusing update from remote peer 38f97f9a1bfa447ea1f8a39f791b7ee2: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:01:51.399824 17664 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260811 09:01:51.400616 17665 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 09:01:51.404183 17397 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.404933 17235 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.408298 17397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.408784 17397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:51.409207 17397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.409652 17397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:51.410056 17397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.410444 17397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:51.410790 17456 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.415081 17490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.415452 17490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:51.416472 17668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:51.417193 17668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:51.420977 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16758
I20260811 09:01:51.443840 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:46395
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:45611
--webserver_port=36587
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:51.888283 17669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:51.888676 17669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:51.888901 17669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:51.897279 17669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:51.897459 17669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:51.897642 17669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:51.908286 17669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:46395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=36587
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:51.909976 17669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:51.911549 17669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:51.929005 17675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:51.930100 17677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:51.930294 17674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:51.931701 17669 server_base.cc:1034] running on GCE node
I20260811 09:01:51.932451 17669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:51.933682 17669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:51.934955 17669 hybrid_clock.cc:648] HybridClock initialized: now 1786438911934901 us; error 50 us; skew 500 ppm
I20260811 09:01:51.935415 17669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:51 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:51.939318 17669 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260811 09:01:51.941335 17669 webserver.cc:460] Webserver started at http://127.9.40.66:36587/ using document root <none> and password file <none>
I20260811 09:01:51.942272 17669 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:51.942461 17669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:51.949131 17669 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:01:51.952634 17687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:51.954505 17669 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:51.954784 17669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "72af8e72ace1481ea8a2f10a379b4729"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "64a2ff5bbb7707e6c628ea0954bbf225"
server_key_iv: "0f20c6dfde15778679b67d4028ec20b0"
server_key_version: "encryptionkey@0"
I20260811 09:01:51.956121 17669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:51.974778 17669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:51.980337 17669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:51.980748 17669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:51.981928 17669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:51.983544 17669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:51.983665 17669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:51.983807 17669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:51.983916 17669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:52.023131 17669 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46395
I20260811 09:01:52.023196 17810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46395 every 8 connection(s)
I20260811 09:01:52.024910 17669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:52.027495 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 17669
I20260811 09:01:52.027815 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16913
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.67@KRBTEST.COM
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438911, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:52.069406 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.029172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.9.40.67:36897)
0811 09:01:52.042124 (+ 12952us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.042137 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.042176 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:01:52.042395 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.042413 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.042798 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.042998 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.045498 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.047003 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.049150 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.050131 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.050292 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.058742 (+ 8450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.058767 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.058792 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.058854 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.063424 (+ 4570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.064762 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.064779 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.064787 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.065064 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.065827 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.065834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.065837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.066993 (+ 1156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.067419 (+ 426us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.068342 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.068777 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12670,"thread_start_us":12523,"threads_started":1}
I20260811 09:01:52.072470 17052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:52.073566 17822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.044229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46063 (local address 127.9.40.126:45611)
0811 09:01:52.044834 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.044840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.045546 (+ 706us) server_negotiation.cc:407] Connection header received
0811 09:01:52.046647 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.046654 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.046746 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.046823 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.050517 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.051745 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.055425 (+ 3680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056183 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056295 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.064544 (+ 8249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.064552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.064557 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.064607 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.068437 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070007 (+ 1570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.070016 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.070019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.070222 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070905 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.070911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.070915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.071318 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.071466 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.072779 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.073161 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":301,"threads_started":1}
I20260811 09:01:52.073721 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.028624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35331 (local address 127.9.40.124:41981)
0811 09:01:52.041380 (+ 12756us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.041387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.043385 (+ 1998us) server_negotiation.cc:407] Connection header received
0811 09:01:52.046686 (+ 3301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.046692 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.046785 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.046876 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.050509 (+ 3633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.051734 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.055505 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056268 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056399 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.063359 (+ 6960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.063369 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.063373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.063436 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.068870 (+ 5434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070540 (+ 1670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.070548 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.070552 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.070778 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.071404 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.071411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.071415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.071899 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.072094 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.072852 (+ 758us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.073206 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12611,"thread_start_us":12773,"threads_started":1}
I20260811 09:01:52.073958 17815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.028479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35903 (local address 127.9.40.125:34221)
0811 09:01:52.041033 (+ 12554us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.041047 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.045686 (+ 4639us) server_negotiation.cc:407] Connection header received
0811 09:01:52.046691 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.046698 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.046785 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.046874 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.050509 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.051744 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.055425 (+ 3681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056157 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056295 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.064231 (+ 7936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.064239 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.064244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.064296 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.068601 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070120 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.070130 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.070135 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.070325 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070905 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.070911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.070915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.071318 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.071466 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.073263 (+ 1797us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.073542 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12354,"thread_start_us":12194,"threads_started":1}
I20260811 09:01:52.075873 17820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.029197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.9.40.66:35903)
0811 09:01:52.045516 (+ 16319us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.045544 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.046265 (+ 721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.047475 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.047490 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.048875 (+ 1385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.050274 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.050285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.052014 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.052019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.055142 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.055169 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056846 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.057486 (+ 631us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.058623 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.058680 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.063910 (+ 5230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.068821 (+ 4911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.068827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.068832 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.069949 (+ 1117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.070471 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070475 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.070478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.070751 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.071587 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.071596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.072937 (+ 1341us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.073160 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.073849 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13536,"mutex_wait_us":874,"spinlock_wait_cycles":43392,"thread_start_us":15282,"threads_started":1}
I20260811 09:01:52.075941 17821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.043892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:46063)
0811 09:01:52.045407 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.045430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.046268 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.047488 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.047494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.048874 (+ 1380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.050248 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.050269 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.052014 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.052019 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.055142 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.055169 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056846 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.057500 (+ 645us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.058642 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.058678 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.064248 (+ 5570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.068726 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.068741 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.068765 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.069812 (+ 1047us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.070402 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070407 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.070413 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.070751 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.071587 (+ 836us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.071596 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.072158 (+ 562us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.074581 (+ 2423us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.075034 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1352,"mutex_wait_us":683,"spinlock_wait_cycles":116864,"thread_start_us":1164,"threads_started":1}
I20260811 09:01:52.076568 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.029697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.9.40.66:35331)
0811 09:01:52.043159 (+ 13462us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.043222 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.046265 (+ 3043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.047475 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.047490 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.048875 (+ 1385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.050227 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.050266 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.052020 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.052028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.055171 (+ 3143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.055194 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.056875 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.056887 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.057513 (+ 626us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.058726 (+ 1213us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.058756 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.062931 (+ 4175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.069118 (+ 6187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.069125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.069130 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.070304 (+ 1174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.070992 (+ 688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.070998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.071003 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.071212 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.072206 (+ 994us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.072212 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.072407 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.074060 (+ 1653us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.074450 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13172,"mutex_wait_us":561,"spinlock_wait_cycles":74752,"thread_start_us":12773,"threads_started":1}
I20260811 09:01:52.078205 17813 heartbeater.cc:347] Connected to a master server at 127.9.40.125:34221
I20260811 09:01:52.078431 17811 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:52.078703 17811 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.078706 17813 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.079397 17812 heartbeater.cc:347] Connected to a master server at 127.9.40.124:41981
I20260811 09:01:52.079631 17812 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.080194 17813 heartbeater.cc:511] Master 127.9.40.125:34221 requested a full tablet report, sending...
I20260811 09:01:52.080345 17812 heartbeater.cc:511] Master 127.9.40.124:41981 requested a full tablet report, sending...
I20260811 09:01:52.081162 17811 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:52.083035 16536 ts_manager.cc:194] Re-registered known tserver with Master: 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395)
I20260811 09:01:52.083173 17446 ts_manager.cc:194] Registered new tserver with Master: 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395)
I20260811 09:01:52.084498 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:46063
I20260811 09:01:52.085075 17225 ts_manager.cc:194] Registered new tserver with Master: 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395)
I20260811 09:01:52.171101 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16913
I20260811 09:01:52.193475 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:36897
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:45611
--webserver_port=35625
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:01:52.343717 17651 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
W20260811 09:01:52.577893 17824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:52.578280 17824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:52.578459 17824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:52.587325 17824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:52.587541 17824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:52.587733 17824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:52.599071 17824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:36897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=35625
--tserver_master_addrs=127.9.40.126:45611,127.9.40.125:34221,127.9.40.124:41981
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:52.600687 17824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:52.602350 17824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:52.617617 17830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:52.617738 17833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:52.617575 17831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:52.618755 17824 server_base.cc:1034] running on GCE node
I20260811 09:01:52.619547 17824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:52.620627 17824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:52.621929 17824 hybrid_clock.cc:648] HybridClock initialized: now 1786438912621874 us; error 45 us; skew 500 ppm
I20260811 09:01:52.622360 17824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438912, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:52.631330 17824 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260811 09:01:52.633436 17824 webserver.cc:460] Webserver started at http://127.9.40.67:35625/ using document root <none> and password file <none>
I20260811 09:01:52.634318 17824 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:52.634508 17824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:52.641355 17824 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:01:52.645056 17844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:52.646783 17824 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:01:52.647047 17824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "994ef811a0e34551a75344f6a17f7197"
format_stamp: "Formatted at 2026-08-11 09:01:48 on dist-test-slave-cbsf"
server_key: "46d9a61f38d5acdb5a29084f6e23c889"
server_key_iv: "f7386e89bf501501f328e4e10b6640a6"
server_key_version: "encryptionkey@0"
I20260811 09:01:52.648280 17824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:52.668185 17824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:52.673441 17824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:52.673719 17824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:52.674839 17824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:01:52.676429 17824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:52.676532 17824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:52.676625 17824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:52.676685 17824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:52.720913 17824 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:36897
I20260811 09:01:52.720984 17967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:36897 every 8 connection(s)
I20260811 09:01:52.722779 17824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:01:52.728135 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 17824
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438912, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438912, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 11 09:01:52 dist-test-slave-cbsf krb5kdc[16481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438912, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260811 09:01:52.784104 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.740080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:42289 (local address 127.9.40.124:41981)
0811 09:01:52.759251 (+ 19171us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.759262 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.762102 (+ 2840us) server_negotiation.cc:407] Connection header received
0811 09:01:52.762343 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.762349 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.762447 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.762591 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.764246 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.765395 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.767330 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.768029 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.768122 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.775079 (+ 6957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.775090 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.775095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.775159 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.779036 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.780402 (+ 1366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.780411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.780417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.780636 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.781493 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.781499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.781503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.781931 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.782080 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.783345 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.783653 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18979,"thread_start_us":18834,"threads_started":1}
I20260811 09:01:52.787173 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.743662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.9.40.67:42289)
0811 09:01:52.761956 (+ 18294us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.761977 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.762177 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.762804 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.762809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.763587 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.763968 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.763983 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.765598 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.765603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.767141 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.767157 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.768509 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.768524 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.769003 (+ 479us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.770060 (+ 1057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.770100 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.774667 (+ 4567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.779256 (+ 4589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.779271 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.779297 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.780204 (+ 907us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.780948 (+ 744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.780953 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.780957 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.781308 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.782207 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.782217 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.782794 (+ 577us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.784473 (+ 1679us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.785295 (+ 822us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18130,"thread_start_us":17948,"threads_started":1}
I20260811 09:01:52.789197 17969 heartbeater.cc:347] Connected to a master server at 127.9.40.124:41981
I20260811 09:01:52.789767 17969 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.791314 17969 heartbeater.cc:511] Master 127.9.40.124:41981 requested a full tablet report, sending...
I20260811 09:01:52.794090 17446 ts_manager.cc:194] Registered new tserver with Master: 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:52.803154 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.726599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43781 (local address 127.9.40.126:45611)
0811 09:01:52.742606 (+ 16007us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.742617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.742657 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:01:52.743901 (+ 1244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.743910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.744026 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.744185 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.758949 (+ 14764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.761682 (+ 2733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.767175 (+ 5493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.767929 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.768103 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.776364 (+ 8261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.776374 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.776382 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.776441 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.795975 (+ 19534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.798742 (+ 2767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.798748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.798751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.798992 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.800285 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.800292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.800296 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.800760 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.800941 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.802399 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.802705 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15814,"thread_start_us":15624,"threads_started":1}
I20260811 09:01:52.803359 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.727138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:43781)
0811 09:01:52.739412 (+ 12274us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.739498 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.743387 (+ 3889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.745028 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.745046 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.755923 (+ 10877us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.757340 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.757365 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.762424 (+ 5059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.762430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.765182 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.765199 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.770409 (+ 5210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.770418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.770612 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.772133 (+ 1521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.772168 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.776019 (+ 3851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.796215 (+ 20196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.796226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.796233 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.797228 (+ 995us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.799855 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.799864 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.799870 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.800118 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.800942 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.800952 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.801261 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.801948 (+ 687us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.802523 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11915,"thread_start_us":11047,"threads_started":1}
I20260811 09:01:52.806808 17968 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45611
I20260811 09:01:52.807037 17968 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.807556 17968 heartbeater.cc:511] Master 127.9.40.126:45611 requested a full tablet report, sending...
I20260811 09:01:52.808785 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.727054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33053 (local address 127.9.40.125:34221)
0811 09:01:52.736955 (+ 9901us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.736967 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.744806 (+ 7839us) server_negotiation.cc:407] Connection header received
0811 09:01:52.744926 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.744934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.745034 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.765492 (+ 20458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.765732 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.766919 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.771103 (+ 4184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.771825 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.785205 (+ 13380us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.785551 (+ 346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.785557 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.785562 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.785601 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.798492 (+ 12891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.801060 (+ 2568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.801066 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.801069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.801310 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.805472 (+ 4162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.805479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.805485 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.806102 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.806283 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.807820 (+ 1537us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.808177 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9662,"thread_start_us":9482,"threads_started":1}
I20260811 09:01:52.809551 16536 ts_manager.cc:194] Re-registered known tserver with Master: 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:52.809636 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.727138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.9.40.67:33053)
0811 09:01:52.744252 (+ 17114us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.744280 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.744451 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.750187 (+ 5736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.750197 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.752651 (+ 2454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:52.753937 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.754012 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.768461 (+ 14449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.768466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.770159 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.770181 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.780483 (+ 10302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.780491 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.780774 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.781455 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:52.781479 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:52.784946 (+ 3467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:52.798735 (+ 13789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.798745 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.798752 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.799635 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.801468 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:52.801477 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:52.801483 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:52.801709 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:52.806471 (+ 4762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:52.806478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:52.806733 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.806933 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.807407 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16886,"mutex_wait_us":69,"spinlock_wait_cycles":17792,"thread_start_us":11460,"threads_started":1}
I20260811 09:01:52.811348 16536 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43781
I20260811 09:01:52.812194 17970 heartbeater.cc:347] Connected to a master server at 127.9.40.125:34221
I20260811 09:01:52.812412 17970 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:52.813072 17970 heartbeater.cc:511] Master 127.9.40.125:34221 requested a full tablet report, sending...
I20260811 09:01:52.814827 17225 ts_manager.cc:194] Registered new tserver with Master: 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:52.824514 17985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.745817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.9.40.126:45611)
0811 09:01:52.761375 (+ 15558us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.761385 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.787571 (+ 26186us) server_negotiation.cc:407] Connection header received
0811 09:01:52.790715 (+ 3144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.790723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.790886 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.790993 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.801042 (+ 10049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.802336 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.805880 (+ 3544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.806611 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.806744 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.815706 (+ 8962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.815713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.815717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.815766 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.820295 (+ 4529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.821570 (+ 1275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.821578 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.821584 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.821817 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.822376 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.822383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.822386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.822882 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.823200 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.823702 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.824012 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15359,"thread_start_us":15148,"threads_started":1}
I20260811 09:01:52.831269 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.800731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.9.40.124:41981)
0811 09:01:52.800939 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.800945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.805691 (+ 4746us) server_negotiation.cc:407] Connection header received
0811 09:01:52.806052 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.806059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.806167 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.806252 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.809592 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.810846 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.817087 (+ 6241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.818035 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.818238 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.822592 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.822600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.822605 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.822667 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.826809 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.827932 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.827940 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.827944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.828182 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.828880 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.828898 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.828901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.829455 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.829764 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.830536 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.830863 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:01:52.835080 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.745817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.9.40.125:34221)
0811 09:01:52.790525 (+ 44708us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.790532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.801556 (+ 11024us) server_negotiation.cc:407] Connection header received
0811 09:01:52.801837 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.801843 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.801928 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.802019 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:52.813128 (+ 11109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.814702 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.817116 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.817682 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.817875 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.821757 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:52.821764 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:52.821768 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:52.821815 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:52.826270 (+ 4455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.827495 (+ 1225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.827503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.827507 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.827718 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:52.828260 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:52.828267 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:52.828270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:52.828816 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:52.829103 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.834034 (+ 4931us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.834378 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44512,"thread_start_us":47610,"threads_started":1}
I20260811 09:01:52.830662 16536 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:52.841317 16536 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:52.916318 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.882265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42820 (local address 127.9.40.67:36897)
0811 09:01:52.890770 (+ 8505us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.890786 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.898683 (+ 7897us) server_negotiation.cc:407] Connection header received
0811 09:01:52.898987 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.899001 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.899478 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.899810 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:52.904661 (+ 4851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.907386 (+ 2725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.911918 (+ 4532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.914155 (+ 2237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.914326 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.914642 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.915618 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.915835 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8022,"thread_start_us":890,"threads_started":1}
I20260811 09:01:52.916313 18001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.881624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36897 (local address 127.0.0.1:42820)
0811 09:01:52.898364 (+ 16740us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.898403 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.898741 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.900936 (+ 2195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.900946 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.901000 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:52.904074 (+ 3074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.904092 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.907553 (+ 3461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.907559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.911589 (+ 4030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.911615 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.915013 (+ 3398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.915022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.915331 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.915383 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.915503 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.915709 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16463,"thread_start_us":16123,"threads_started":1}
I20260811 09:01:52.918249 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.883683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46395 (local address 127.0.0.1:36698)
0811 09:01:52.903370 (+ 19687us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.903389 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.903634 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.905334 (+ 1700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.905340 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.905368 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:52.906171 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.906207 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.908649 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.908655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.911848 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.911872 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.916816 (+ 4944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.916826 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.917103 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.917188 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.917308 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.917489 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19494,"mutex_wait_us":2,"thread_start_us":19235,"threads_started":1}
I20260811 09:01:52.918448 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.884643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42497 (local address 127.0.0.1:48750)
0811 09:01:52.904664 (+ 20021us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:52.904716 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:52.904997 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:52.906025 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:52.906030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:52.906067 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:52.907073 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.907089 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.909713 (+ 2624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.909720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.913478 (+ 3758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:52.913502 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.917182 (+ 3680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.917189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:52.917456 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.917490 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:01:52.917591 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:01:52.917801 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19785,"mutex_wait_us":30,"thread_start_us":19390,"threads_started":1}
I20260811 09:01:52.919389 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.885087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.9.40.65:42497)
0811 09:01:52.900633 (+ 15546us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.900650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.904853 (+ 4203us) server_negotiation.cc:407] Connection header received
0811 09:01:52.905207 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.905214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.905502 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.905785 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:52.907401 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.909529 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.913801 (+ 4272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.916401 (+ 2600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.916542 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.916883 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.917689 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.917933 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15183,"thread_start_us":15582,"threads_started":1}
I20260811 09:01:52.919502 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:52.883897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.9.40.66:46395)
0811 09:01:52.903194 (+ 19297us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:52.903206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:52.903527 (+ 321us) server_negotiation.cc:407] Connection header received
0811 09:01:52.904643 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:52.904653 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:52.905083 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:52.905367 (+ 284us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:52.906445 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.908375 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.912103 (+ 3728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:52.914158 (+ 2055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:52.914355 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:52.914720 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:52.918782 (+ 4062us) server_negotiation.cc:299] Negotiation successful
0811 09:01:52.919047 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18939,"thread_start_us":17941,"threads_started":1}
I20260811 09:01:52.922224 17894 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0d09b4655d46d99e0457e971f0164a (DEFAULT_TABLE table=test-table [id=0ebe71ce4c08493da4dc73e557da7b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:52.924500 17574 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0d09b4655d46d99e0457e971f0164a (DEFAULT_TABLE table=test-table [id=0ebe71ce4c08493da4dc73e557da7b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:52.925246 17894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0d09b4655d46d99e0457e971f0164a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:52.925218 17739 tablet_service.cc:1467] Processing CreateTablet for tablet 2c0d09b4655d46d99e0457e971f0164a (DEFAULT_TABLE table=test-table [id=0ebe71ce4c08493da4dc73e557da7b49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:52.926409 17574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0d09b4655d46d99e0457e971f0164a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:52.926787 17739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c0d09b4655d46d99e0457e971f0164a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:52.946859 18007 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Bootstrap starting.
I20260811 09:01:52.950114 18008 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Bootstrap starting.
I20260811 09:01:52.952924 18007 tablet_bootstrap.cc:654] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:52.953364 18010 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Bootstrap starting.
I20260811 09:01:52.955606 18007 log.cc:824] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:52.957505 18008 tablet_bootstrap.cc:654] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:52.959617 18008 log.cc:824] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:52.959671 18010 tablet_bootstrap.cc:654] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:52.962302 18007 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: No bootstrap required, opened a new log
I20260811 09:01:52.962227 18010 log.cc:824] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:52.962695 18007 ts_tablet_manager.cc:1397] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20260811 09:01:52.965356 18008 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: No bootstrap required, opened a new log
I20260811 09:01:52.965766 18008 ts_tablet_manager.cc:1397] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.008s
I20260811 09:01:52.966722 18010 tablet_bootstrap.cc:492] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: No bootstrap required, opened a new log
I20260811 09:01:52.967077 18010 ts_tablet_manager.cc:1397] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260811 09:01:52.978739 18007 raft_consensus.cc:348] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.979249 18010 raft_consensus.cc:348] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.979676 18007 raft_consensus.cc:374] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:52.979799 18010 raft_consensus.cc:374] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:52.979897 18007 raft_consensus.cc:729] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf48d96894746e08e1ca33b78bea683, State: Initialized, Role: FOLLOWER
I20260811 09:01:52.980003 18010 raft_consensus.cc:729] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72af8e72ace1481ea8a2f10a379b4729, State: Initialized, Role: FOLLOWER
I20260811 09:01:52.980947 18010 consensus_queue.cc:260] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.980986 18007 consensus_queue.cc:260] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.982687 17811 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
I20260811 09:01:52.982420 18008 raft_consensus.cc:348] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.983204 18008 raft_consensus.cc:374] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:52.983367 18008 raft_consensus.cc:729] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 994ef811a0e34551a75344f6a17f7197, State: Initialized, Role: FOLLOWER
I20260811 09:01:52.983469 18010 ts_tablet_manager.cc:1428] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Time spent starting tablet: real 0.016s user 0.005s sys 0.012s
I20260811 09:01:52.984544 18008 consensus_queue.cc:260] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:52.990759 18007 ts_tablet_manager.cc:1428] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Time spent starting tablet: real 0.028s user 0.017s sys 0.010s
I20260811 09:01:52.993487 17968 heartbeater.cc:503] Master 127.9.40.126:45611 was elected leader, sending a full tablet report...
I20260811 09:01:52.994720 18008 ts_tablet_manager.cc:1428] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Time spent starting tablet: real 0.029s user 0.013s sys 0.014s
W20260811 09:01:53.028503 17814 tablet.cc:2369] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:01:53.039714 17660 tablet.cc:2369] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:53.073165 18017 raft_consensus.cc:482] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:53.073765 18017 raft_consensus.cc:504] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:53.076270 18017 leader_election.cc:288] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395), bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:53.084605 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.076549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46395 (local address 127.9.40.67:59577)
0811 09:01:53.076925 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:53.076948 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:53.077118 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:53.077680 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:53.077687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:53.077715 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:53.078062 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:53.078076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.079375 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.079384 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:53.081720 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:53.081743 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.083409 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.083418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:53.083943 (+ 525us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.083977 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:01:53.084074 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:01:53.084202 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":26}
I20260811 09:01:53.085031 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.076641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59577 (local address 127.9.40.66:46395)
0811 09:01:53.076928 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.076936 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.077083 (+ 147us) server_negotiation.cc:407] Connection header received
0811 09:01:53.077369 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.077378 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.077464 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.077566 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:53.078239 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.079189 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.081992 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.083143 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.083294 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.083379 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.084322 (+ 943us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.084530 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 09:01:53.085366 17974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.076997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42497 (local address 127.9.40.67:41233)
0811 09:01:53.077365 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:53.077383 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:53.077464 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:53.077877 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:53.077880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:53.077905 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:53.078083 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:53.078091 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.079478 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.079488 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:53.081726 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:53.081743 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.083903 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.083914 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:53.084569 (+ 655us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.084614 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 09:01:53.084822 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 09:01:53.084982 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":2}
I20260811 09:01:53.085960 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.077112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41233 (local address 127.9.40.65:42497)
0811 09:01:53.077389 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.077395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.077442 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:01:53.077660 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.077668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.077755 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.077862 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:53.078326 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.079305 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.082005 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.083423 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.083659 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.083851 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.085396 (+ 1545us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.085637 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 09:01:53.086758 17760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0d09b4655d46d99e0457e971f0164a" candidate_uuid: "994ef811a0e34551a75344f6a17f7197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72af8e72ace1481ea8a2f10a379b4729" is_pre_election: true
I20260811 09:01:53.087266 17597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0d09b4655d46d99e0457e971f0164a" candidate_uuid: "994ef811a0e34551a75344f6a17f7197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf48d96894746e08e1ca33b78bea683" is_pre_election: true
I20260811 09:01:53.087520 17760 raft_consensus.cc:2454] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 994ef811a0e34551a75344f6a17f7197 in term 0.
I20260811 09:01:53.087965 17597 raft_consensus.cc:2454] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 994ef811a0e34551a75344f6a17f7197 in term 0.
I20260811 09:01:53.088641 17845 leader_election.cc:302] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72af8e72ace1481ea8a2f10a379b4729, 994ef811a0e34551a75344f6a17f7197; no voters:
I20260811 09:01:53.089248 18017 raft_consensus.cc:2781] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:53.089430 18017 raft_consensus.cc:482] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:53.089582 18017 raft_consensus.cc:3037] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:53.093243 18017 raft_consensus.cc:504] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:53.094198 18017 leader_election.cc:288] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [CANDIDATE]: Term 1 election: Requested vote from peers 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395), bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497)
I20260811 09:01:53.095139 17597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0d09b4655d46d99e0457e971f0164a" candidate_uuid: "994ef811a0e34551a75344f6a17f7197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf48d96894746e08e1ca33b78bea683"
I20260811 09:01:53.095148 17760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c0d09b4655d46d99e0457e971f0164a" candidate_uuid: "994ef811a0e34551a75344f6a17f7197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72af8e72ace1481ea8a2f10a379b4729"
I20260811 09:01:53.095450 17760 raft_consensus.cc:3037] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:53.095536 17597 raft_consensus.cc:3037] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:53.100656 17597 raft_consensus.cc:2454] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 994ef811a0e34551a75344f6a17f7197 in term 1.
I20260811 09:01:53.100633 17760 raft_consensus.cc:2454] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 994ef811a0e34551a75344f6a17f7197 in term 1.
I20260811 09:01:53.101521 17845 leader_election.cc:302] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 72af8e72ace1481ea8a2f10a379b4729, 994ef811a0e34551a75344f6a17f7197, bcf48d96894746e08e1ca33b78bea683; no voters:
I20260811 09:01:53.102121 18017 raft_consensus.cc:2781] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:53.103003 18017 raft_consensus.cc:686] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 1 LEADER]: Becoming Leader. State: Replica: 994ef811a0e34551a75344f6a17f7197, State: Running, Role: LEADER
I20260811 09:01:53.103924 18017 consensus_queue.cc:237] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } }
I20260811 09:01:53.112074 16534 catalog_manager.cc:5495] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 reported cstate change: term changed from 0 to 1, leader changed from <none> to 994ef811a0e34551a75344f6a17f7197 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "994ef811a0e34551a75344f6a17f7197" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } health_report { overall_health: UNKNOWN } } }
I20260811 09:01:53.169937 18002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.162478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.9.40.67:36897)
0811 09:01:53.162682 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.162687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.162768 (+ 81us) server_negotiation.cc:407] Connection header received
0811 09:01:53.162993 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.162997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.163094 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.163280 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:53.163765 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.164647 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.166468 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.167134 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.167259 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.168140 (+ 881us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:53.168862 (+ 722us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:53.169002 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.169314 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.169437 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:01:53.182296 17760 raft_consensus.cc:1261] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Refusing update from remote peer 994ef811a0e34551a75344f6a17f7197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:53.182296 17597 raft_consensus.cc:1261] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Refusing update from remote peer 994ef811a0e34551a75344f6a17f7197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:53.184036 18018 consensus_queue.cc:1035] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 09:01:53.184808 18017 consensus_queue.cc:1035] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:53.233729 16534 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:53.234088 16534 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:53.242722 16534 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 38f97f9a1bfa447ea1f8a39f791b7ee2: Sending DeleteTablet for 3 replicas of tablet 2c0d09b4655d46d99e0457e971f0164a
I20260811 09:01:53.244235 17894 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0d09b4655d46d99e0457e971f0164a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:53 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42820
I20260811 09:01:53.244870 17739 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0d09b4655d46d99e0457e971f0164a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:53 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:36698
I20260811 09:01:53.247788 18031 tablet_replica.cc:330] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: stopping tablet replica
I20260811 09:01:53.248080 17574 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c0d09b4655d46d99e0457e971f0164a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:53 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48750
I20260811 09:01:53.256296 18032 tablet_replica.cc:330] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: stopping tablet replica
I20260811 09:01:53.257329 18032 raft_consensus.cc:2229] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:53.258399 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 16498
I20260811 09:01:53.258566 18032 raft_consensus.cc:2258] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:53.260080 18031 raft_consensus.cc:2229] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:53.265462 16569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:53.274027 18030 tablet_replica.cc:330] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: stopping tablet replica
I20260811 09:01:53.276280 18031 raft_consensus.cc:2258] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:53.277112 18030 raft_consensus.cc:2229] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:53.278656 18030 raft_consensus.cc:2258] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:53.282835 18030 ts_tablet_manager.cc:1905] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:53.283131 18031 ts_tablet_manager.cc:1905] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:53.289561 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.252752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.9.40.125:34221)
0811 09:01:53.253333 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.253346 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.257172 (+ 3826us) server_negotiation.cc:407] Connection header received
0811 09:01:53.258609 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.258617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.258743 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.258896 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:53.263886 (+ 4990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.265451 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.268151 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.269252 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.269398 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.278259 (+ 8861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:53.278270 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:53.278276 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:53.278352 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:53.284527 (+ 6175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.286000 (+ 1473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.286008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.286013 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.286268 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.286988 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.286995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.286999 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.287517 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:53.287813 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.288650 (+ 837us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.289026 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:01:53.289891 18032 ts_tablet_manager.cc:1905] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:53.311430 18030 ts_tablet_manager.cc:1918] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:53.311712 18030 log.cc:1196] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/2c0d09b4655d46d99e0457e971f0164a
I20260811 09:01:53.312593 18030 ts_tablet_manager.cc:1939] T 2c0d09b4655d46d99e0457e971f0164a P 994ef811a0e34551a75344f6a17f7197: Deleting consensus metadata
I20260811 09:01:53.314989 18031 ts_tablet_manager.cc:1918] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:53.315433 18031 log.cc:1196] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/2c0d09b4655d46d99e0457e971f0164a
I20260811 09:01:53.316215 18031 ts_tablet_manager.cc:1939] T 2c0d09b4655d46d99e0457e971f0164a P 72af8e72ace1481ea8a2f10a379b4729: Deleting consensus metadata
I20260811 09:01:53.320866 18032 ts_tablet_manager.cc:1918] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:53.321233 18032 log.cc:1196] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/2c0d09b4655d46d99e0457e971f0164a
I20260811 09:01:53.322193 18032 ts_tablet_manager.cc:1939] T 2c0d09b4655d46d99e0457e971f0164a P bcf48d96894746e08e1ca33b78bea683: Deleting consensus metadata
I20260811 09:01:53.323925 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.257832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.9.40.124:41981)
0811 09:01:53.258077 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.258084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.293072 (+ 34988us) server_negotiation.cc:407] Connection header received
0811 09:01:53.293590 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.293597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.293698 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.293786 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:53.297470 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.299060 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.304222 (+ 5162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.305439 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.305579 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.311625 (+ 6046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:53.311636 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:53.311643 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:53.311700 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:53.317083 (+ 5383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.318975 (+ 1892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.318984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.318990 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.319342 (+ 352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.320136 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.320156 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.320159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.320732 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:53.321215 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.323011 (+ 1796us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.323342 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 09:01:53.451366 16518 catalog_manager.cc:4841] TS 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395): tablet 2c0d09b4655d46d99e0457e971f0164a (table test-table [id=0ebe71ce4c08493da4dc73e557da7b49]) successfully deleted
I20260811 09:01:53.452116 16519 catalog_manager.cc:4841] TS bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497): tablet 2c0d09b4655d46d99e0457e971f0164a (table test-table [id=0ebe71ce4c08493da4dc73e557da7b49]) successfully deleted
I20260811 09:01:53.452559 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 16498
I20260811 09:01:53.521886 17987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.495675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.9.40.125:34221)
0811 09:01:53.495959 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.495969 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.497232 (+ 1263us) server_negotiation.cc:407] Connection header received
0811 09:01:53.497626 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.497636 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.497762 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.497862 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:53.501319 (+ 3457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.503016 (+ 1697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.506556 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.507996 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.508187 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.513287 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:53.513295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:53.513300 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:53.513358 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:53.518059 (+ 4701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.519083 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.519091 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.519095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.519300 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.519910 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.519918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.519922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.520392 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:53.520762 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.521188 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.521477 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 09:01:53.528546 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:53.496877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37058 (local address 127.9.40.124:41981)
0811 09:01:53.497131 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:53.497161 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:53.511089 (+ 13928us) server_negotiation.cc:407] Connection header received
0811 09:01:53.511322 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:53.511328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:53.511430 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:53.511568 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:53.513627 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.514648 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.516658 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:53.517423 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:53.517678 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:53.521626 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:53.521637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:53.521642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:53.521693 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:53.525072 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.526179 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.526184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.526186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.526342 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:53.526832 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:53.526835 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:53.526837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:53.527173 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:53.527379 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:53.527869 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 09:01:53.528109 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:01:54.217058 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.216129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:60573)
0811 09:01:54.216645 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.216826 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":286,"threads_started":1}
W20260811 09:01:54.217756 17651 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:45611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.221148 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.220344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:50817)
0811 09:01:54.220836 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.220963 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":241,"threads_started":1}
W20260811 09:01:54.221621 17811 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:45611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.226711 18045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.225839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:35849)
0811 09:01:54.226383 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.226524 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":294,"threads_started":1}
W20260811 09:01:54.227099 17968 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:45611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.744220 18046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38f97f9a1bfa447ea1f8a39f791b7ee2)
I20260811 09:01:54.744608 18046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } }
I20260811 09:01:54.746098 18047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38f97f9a1bfa447ea1f8a39f791b7ee2)
I20260811 09:01:54.746281 18046 leader_election.cc:288] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611), 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125:34221)
I20260811 09:01:54.746459 18047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } }
I20260811 09:01:54.747699 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.746373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34962)
0811 09:01:54.747126 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.747397 (+ 271us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":318,"threads_started":1}
W20260811 09:01:54.748154 17428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.748526 18047 leader_election.cc:288] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611), 086f89fe0d4e44a1aa325df64a13e68e (127.9.40.124:41981)
I20260811 09:01:54.750792 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.749798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34966)
0811 09:01:54.750346 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.750507 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358}
I20260811 09:01:54.750836 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.748856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34964)
0811 09:01:54.750303 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.750541 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1336,"thread_start_us":484,"threads_started":1}
W20260811 09:01:54.751374 17209 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
W20260811 09:01:54.752128 17428 leader_election.cc:334] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611): Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.755144 18055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.754602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34982)
0811 09:01:54.754812 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.754933 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:01:54.755501 17209 leader_election.cc:334] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611): Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.756413 18053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.746912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.9.40.125:34221)
0811 09:01:54.748054 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:54.748060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:54.748076 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:01:54.748139 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:54.748142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:54.748205 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:54.748333 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:54.749065 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.750409 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.753751 (+ 3342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.755107 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.755263 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.755391 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:54.755831 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 09:01:54.755994 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":996,"thread_start_us":777,"threads_started":1}
I20260811 09:01:54.756428 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.746560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:34221 (local address 127.0.0.1:33970)
0811 09:01:54.747260 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.747283 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:54.747463 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:54.748358 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:54.748363 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:54.748391 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:54.748818 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.748836 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.750648 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.750655 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.753528 (+ 2873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.753549 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.755377 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.755384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.755626 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.755679 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 09:01:54.755798 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:01:54.755980 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":532,"threads_started":1}
I20260811 09:01:54.757351 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "086f89fe0d4e44a1aa325df64a13e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9c3335d4282f461481d2a2fb9631abc4" is_pre_election: true
I20260811 09:01:54.757750 17235 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 086f89fe0d4e44a1aa325df64a13e68e in term 1.
I20260811 09:01:54.758365 17432 leader_election.cc:302] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 086f89fe0d4e44a1aa325df64a13e68e, 9c3335d4282f461481d2a2fb9631abc4; no voters: 38f97f9a1bfa447ea1f8a39f791b7ee2
I20260811 09:01:54.758844 18046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 09:01:54.759058 18046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 38f97f9a1bfa447ea1f8a39f791b7ee2)
I20260811 09:01:54.759173 18046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:01:54.759631 18056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.749781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:41981 (local address 127.0.0.1:37066)
0811 09:01:54.751309 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.751338 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:54.751528 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:54.752776 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:54.752785 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:54.752827 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:54.753329 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.753347 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.755262 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.755267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.757418 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.757434 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.758788 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.758794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.758978 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.759012 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:01:54.759094 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:01:54.759250 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1315,"thread_start_us":248,"threads_started":1}
I20260811 09:01:54.759788 18057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.750030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.9.40.124:41981)
0811 09:01:54.752295 (+ 2265us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:54.752302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:54.752332 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:01:54.752412 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:54.752417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:54.752513 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:54.752969 (+ 456us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:54.753558 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.755034 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.757601 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.758610 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.758940 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.759203 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:54.759257 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 09:01:54.759410 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2076,"thread_start_us":1861,"threads_started":1}
I20260811 09:01:54.760447 17456 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9c3335d4282f461481d2a2fb9631abc4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "086f89fe0d4e44a1aa325df64a13e68e" is_pre_election: true
I20260811 09:01:54.762457 18046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } }
I20260811 09:01:54.763147 17456 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9c3335d4282f461481d2a2fb9631abc4 in current term 2: Already voted for candidate 086f89fe0d4e44a1aa325df64a13e68e in this term.
I20260811 09:01:54.764114 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.763558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34986)
0811 09:01:54.763771 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.763906 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:01:54.764118 18046 leader_election.cc:288] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 election: Requested vote from peers 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611), 9c3335d4282f461481d2a2fb9631abc4 (127.9.40.125:34221)
I20260811 09:01:54.764329 17209 leader_election.cc:302] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9c3335d4282f461481d2a2fb9631abc4; no voters: 086f89fe0d4e44a1aa325df64a13e68e, 38f97f9a1bfa447ea1f8a39f791b7ee2
I20260811 09:01:54.764787 18047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:01:54.764781 17235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "086f89fe0d4e44a1aa325df64a13e68e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9c3335d4282f461481d2a2fb9631abc4"
I20260811 09:01:54.765686 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.765276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:34994)
0811 09:01:54.765457 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.765548 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:01:54.765869 17428 leader_election.cc:334] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611): Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
I20260811 09:01:54.767809 18047 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:01:54.770829 17235 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 086f89fe0d4e44a1aa325df64a13e68e in term 2.
I20260811 09:01:54.771829 17432 leader_election.cc:302] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 086f89fe0d4e44a1aa325df64a13e68e, 9c3335d4282f461481d2a2fb9631abc4; no voters: 38f97f9a1bfa447ea1f8a39f791b7ee2
I20260811 09:01:54.772298 18046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:01:54.773062 18046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [term 2 LEADER]: Becoming Leader. State: Replica: 086f89fe0d4e44a1aa325df64a13e68e, State: Running, Role: LEADER
I20260811 09:01:54.773512 18046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } }
I20260811 09:01:54.775728 18059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086f89fe0d4e44a1aa325df64a13e68e. Latest consensus state: current_term: 2 leader_uuid: "086f89fe0d4e44a1aa325df64a13e68e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:54.776218 18059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:54.777056 18061 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:54.781647 18061 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:54.782891 18061 catalog_manager.cc:1259] Loaded cluster ID: 49232210f17d47eb9a822f81a921a51e
I20260811 09:01:54.783021 18061 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:54.784341 18061 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:54.785317 18061 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Loaded TSK: 0
I20260811 09:01:54.786399 18061 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:01:54.799063 17444 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:01:54.807075 17444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:01:54.812068 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.811595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:35016)
0811 09:01:54.811813 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.811901 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 09:01:54.812240 17235 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [term 2 FOLLOWER]: Refusing update from remote peer 086f89fe0d4e44a1aa325df64a13e68e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 09:01:54.813469 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.812908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:35024)
0811 09:01:54.813161 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.813264 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 09:01:54.813287 18059 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 09:01:54.813668 17428 consensus_peers.cc:487] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e -> Peer 38f97f9a1bfa447ea1f8a39f791b7ee2 (127.9.40.126:45611): Couldn't send request to peer 38f97f9a1bfa447ea1f8a39f791b7ee2. Status: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:01:54.816972 18047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086f89fe0d4e44a1aa325df64a13e68e. Latest consensus state: current_term: 2 leader_uuid: "086f89fe0d4e44a1aa325df64a13e68e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:54.817390 18047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:54.819345 18046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "086f89fe0d4e44a1aa325df64a13e68e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:54.819778 18046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:54.820636 18047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "086f89fe0d4e44a1aa325df64a13e68e" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "38f97f9a1bfa447ea1f8a39f791b7ee2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45611 } } peers { permanent_uuid: "9c3335d4282f461481d2a2fb9631abc4" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 34221 } attrs { promote: false } } peers { permanent_uuid: "086f89fe0d4e44a1aa325df64a13e68e" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 41981 } attrs { promote: false } } }
I20260811 09:01:54.821007 18047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c3335d4282f461481d2a2fb9631abc4 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:01:54.837796 18049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.829998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42497 (local address 127.0.0.1:48752)
0811 09:01:54.830218 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.830246 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:54.830386 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:54.830926 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:54.830934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:54.830970 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:54.831314 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.831332 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.833043 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.833052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.834301 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.834315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.836628 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.836637 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.836964 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.837003 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:01:54.837125 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 09:01:54.837342 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:01:54.838624 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.830002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.9.40.65:42497)
0811 09:01:54.830567 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:54.830573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:54.830592 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:01:54.830669 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:54.830674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:54.830768 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:54.830885 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:54.831530 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.832803 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.834493 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.836287 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.836524 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.836724 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:54.837203 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 09:01:54.838236 (+ 1033us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":291,"threads_started":1}
I20260811 09:01:54.839644 17574 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba3dafba7df45338862f49cf1670676 (DEFAULT_TABLE table=test-table [id=3050755a2b234d219a1d263558252ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:54.840269 17574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba3dafba7df45338862f49cf1670676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:54.844861 18070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.831823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36897 (local address 127.0.0.1:42834)
0811 09:01:54.837153 (+ 5330us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.837183 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:54.837408 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:54.838124 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:54.838129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:54.838166 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:54.838585 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.838598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.840024 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.840030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.842121 (+ 2091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.842134 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.843840 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.843852 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.844103 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.844139 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:01:54.844254 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 09:01:54.844424 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5131,"thread_start_us":4919,"threads_started":1}
I20260811 09:01:54.845305 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.831228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.9.40.67:36897)
0811 09:01:54.833022 (+ 1794us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:54.833029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:54.837281 (+ 4252us) server_negotiation.cc:407] Connection header received
0811 09:01:54.837609 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:54.837617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:54.837715 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:54.837806 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:54.838776 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.839896 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.842308 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.843548 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.843724 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.843899 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:54.844320 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 09:01:54.844519 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1648,"thread_start_us":1540,"threads_started":1}
I20260811 09:01:54.846227 17894 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba3dafba7df45338862f49cf1670676 (DEFAULT_TABLE table=test-table [id=3050755a2b234d219a1d263558252ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:54.846796 17894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba3dafba7df45338862f49cf1670676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:54.846951 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.831701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46395 (local address 127.0.0.1:36704)
0811 09:01:54.832171 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.832197 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:54.832359 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:54.839573 (+ 7214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:54.839578 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:54.839611 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:54.839903 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.839918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.841666 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.841673 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.843947 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:54.843967 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.846091 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.846099 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:54.846320 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.846360 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:01:54.846475 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 09:01:54.846664 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317}
I20260811 09:01:54.847743 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.832133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.9.40.66:46395)
0811 09:01:54.839164 (+ 7031us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:54.839171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:54.839191 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:01:54.839265 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:54.839270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:54.839359 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:54.839489 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:54.840122 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.841207 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.844167 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:54.845811 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:54.845971 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:54.846083 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:54.847212 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0811 09:01:54.847395 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6866,"thread_start_us":7066,"threads_started":1}
I20260811 09:01:54.848232 18071 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Bootstrap starting.
I20260811 09:01:54.848577 17739 tablet_service.cc:1467] Processing CreateTablet for tablet 7ba3dafba7df45338862f49cf1670676 (DEFAULT_TABLE table=test-table [id=3050755a2b234d219a1d263558252ada]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:01:54.849077 17739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ba3dafba7df45338862f49cf1670676. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:54.850297 18071 tablet_bootstrap.cc:654] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:54.854650 18073 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Bootstrap starting.
I20260811 09:01:54.854722 18071 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: No bootstrap required, opened a new log
I20260811 09:01:54.854910 18071 ts_tablet_manager.cc:1397] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260811 09:01:54.855875 18071 raft_consensus.cc:348] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.856227 18071 raft_consensus.cc:374] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:54.856330 18071 raft_consensus.cc:729] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcf48d96894746e08e1ca33b78bea683, State: Initialized, Role: FOLLOWER
I20260811 09:01:54.856621 18074 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Bootstrap starting.
I20260811 09:01:54.856621 18071 consensus_queue.cc:260] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.857406 18073 tablet_bootstrap.cc:654] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:54.859264 18074 tablet_bootstrap.cc:654] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:54.860061 18073 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: No bootstrap required, opened a new log
I20260811 09:01:54.860252 18073 ts_tablet_manager.cc:1397] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 09:01:54.861254 18073 raft_consensus.cc:348] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.861574 18073 raft_consensus.cc:374] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:54.861707 18073 raft_consensus.cc:729] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 994ef811a0e34551a75344f6a17f7197, State: Initialized, Role: FOLLOWER
I20260811 09:01:54.862020 18073 consensus_queue.cc:260] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.862529 18074 tablet_bootstrap.cc:492] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: No bootstrap required, opened a new log
I20260811 09:01:54.862762 18074 ts_tablet_manager.cc:1397] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 09:01:54.863016 18073 ts_tablet_manager.cc:1428] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:54.864764 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.863581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:56211)
0811 09:01:54.864240 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.864433 (+ 193us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":333,"threads_started":1}
I20260811 09:01:54.866004 18071 ts_tablet_manager.cc:1428] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20260811 09:01:54.867522 17969 heartbeater.cc:503] Master 127.9.40.124:41981 was elected leader, sending a full tablet report...
I20260811 09:01:54.867955 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.866663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:45467)
0811 09:01:54.867307 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.867558 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":326,"threads_started":1}
I20260811 09:01:54.867853 18074 raft_consensus.cc:348] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.868160 18074 raft_consensus.cc:374] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:54.868271 18074 raft_consensus.cc:729] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72af8e72ace1481ea8a2f10a379b4729, State: Initialized, Role: FOLLOWER
I20260811 09:01:54.869673 18074 consensus_queue.cc:260] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:54.873008 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.871790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:54875)
0811 09:01:54.872420 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.872623 (+ 203us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":671,"threads_started":1}
I20260811 09:01:54.870687 18074 ts_tablet_manager.cc:1428] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:01:54.875545 17812 heartbeater.cc:503] Master 127.9.40.124:41981 was elected leader, sending a full tablet report...
I20260811 09:01:54.876627 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.876176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:56127)
0811 09:01:54.876362 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.876475 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:01:54.879703 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.866194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:33759)
0811 09:01:54.879335 (+ 13141us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.879471 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12998,"threads_started":1}
I20260811 09:01:54.881292 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:54.880890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:49367)
0811 09:01:54.881070 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:54.881156 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:01:54.882138 17652 heartbeater.cc:503] Master 127.9.40.124:41981 was elected leader, sending a full tablet report...
W20260811 09:01:54.984383 17971 tablet.cc:2369] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:01:55.007670 18083 raft_consensus.cc:482] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:01:55.007997 18083 raft_consensus.cc:504] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:55.009828 18083 leader_election.cc:288] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497), 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:55.016278 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.010329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:47325 (local address 127.9.40.67:36897)
0811 09:01:55.010566 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.010572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.010693 (+ 121us) server_negotiation.cc:407] Connection header received
0811 09:01:55.010990 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.010996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.011086 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.011208 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:55.011712 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.012455 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.014311 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.015112 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.015271 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.015384 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.015708 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.015860 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:01:55.016323 18085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.010176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36897 (local address 127.9.40.66:47325)
0811 09:01:55.010544 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.010568 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:55.010785 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:55.011231 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:55.011236 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:55.011264 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:55.011564 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:55.011574 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.012598 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.012605 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:55.014132 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:55.014146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.015354 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.015359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:55.015533 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.015558 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:01:55.015668 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 09:01:55.015835 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232}
I20260811 09:01:55.016855 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.010330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59775 (local address 127.9.40.65:42497)
0811 09:01:55.010599 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.010609 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.011188 (+ 579us) server_negotiation.cc:407] Connection header received
0811 09:01:55.011404 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.011408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.011471 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.011575 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:01:55.012105 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.012929 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.014678 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.015639 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.015790 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.015916 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.016391 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.016574 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:01:55.017277 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.010234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42497 (local address 127.9.40.66:59775)
0811 09:01:55.011073 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.011087 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:55.011238 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:55.011597 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:55.011601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:55.011624 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:01:55.011951 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:55.011960 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.013063 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.013071 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:55.014493 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:55.014505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.015881 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.015887 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:55.016182 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.016210 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:01:55.016360 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:01:55.016933 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":522,"threads_started":1}
I20260811 09:01:55.017397 17916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba3dafba7df45338862f49cf1670676" candidate_uuid: "72af8e72ace1481ea8a2f10a379b4729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "994ef811a0e34551a75344f6a17f7197" is_pre_election: true
I20260811 09:01:55.017890 17916 raft_consensus.cc:2454] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72af8e72ace1481ea8a2f10a379b4729 in term 0.
I20260811 09:01:55.017987 17597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba3dafba7df45338862f49cf1670676" candidate_uuid: "72af8e72ace1481ea8a2f10a379b4729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf48d96894746e08e1ca33b78bea683" is_pre_election: true
I20260811 09:01:55.018249 17597 raft_consensus.cc:2454] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72af8e72ace1481ea8a2f10a379b4729 in term 0.
I20260811 09:01:55.018677 17690 leader_election.cc:302] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72af8e72ace1481ea8a2f10a379b4729, 994ef811a0e34551a75344f6a17f7197; no voters:
I20260811 09:01:55.019131 18083 raft_consensus.cc:2781] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:01:55.019289 18083 raft_consensus.cc:482] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:01:55.019407 18083 raft_consensus.cc:3037] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:55.021957 18083 raft_consensus.cc:504] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:55.022809 18083 leader_election.cc:288] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [CANDIDATE]: Term 1 election: Requested vote from peers bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497), 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897)
I20260811 09:01:55.023499 17597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba3dafba7df45338862f49cf1670676" candidate_uuid: "72af8e72ace1481ea8a2f10a379b4729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcf48d96894746e08e1ca33b78bea683"
I20260811 09:01:55.023533 17916 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ba3dafba7df45338862f49cf1670676" candidate_uuid: "72af8e72ace1481ea8a2f10a379b4729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "994ef811a0e34551a75344f6a17f7197"
I20260811 09:01:55.023753 17597 raft_consensus.cc:3037] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:55.023787 17916 raft_consensus.cc:3037] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:55.026230 17597 raft_consensus.cc:2454] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72af8e72ace1481ea8a2f10a379b4729 in term 1.
I20260811 09:01:55.026880 17689 leader_election.cc:302] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72af8e72ace1481ea8a2f10a379b4729, bcf48d96894746e08e1ca33b78bea683; no voters:
I20260811 09:01:55.027179 17916 raft_consensus.cc:2454] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72af8e72ace1481ea8a2f10a379b4729 in term 1.
I20260811 09:01:55.027304 18083 raft_consensus.cc:2781] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:55.027911 18083 raft_consensus.cc:686] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 1 LEADER]: Becoming Leader. State: Replica: 72af8e72ace1481ea8a2f10a379b4729, State: Running, Role: LEADER
I20260811 09:01:55.029101 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.028571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:59787)
0811 09:01:55.028789 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.028909 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:01:55.028913 18083 consensus_queue.cc:237] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } }
I20260811 09:01:55.033986 17446 catalog_manager.cc:5495] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "72af8e72ace1481ea8a2f10a379b4729" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72af8e72ace1481ea8a2f10a379b4729" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46395 } health_report { overall_health: HEALTHY } } }
I20260811 09:01:55.099181 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.092896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.9.40.66:46395)
0811 09:01:55.093109 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.093114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.093285 (+ 171us) server_negotiation.cc:407] Connection header received
0811 09:01:55.093468 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.093474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.093601 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.093944 (+ 343us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:01:55.094371 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.095249 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.096672 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.097211 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.097328 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.097828 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:01:55.098472 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:01:55.098598 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.098793 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.098909 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:01:55.104514 17597 raft_consensus.cc:1261] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Refusing update from remote peer 72af8e72ace1481ea8a2f10a379b4729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:55.104910 17916 raft_consensus.cc:1261] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Refusing update from remote peer 72af8e72ace1481ea8a2f10a379b4729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:01:55.105490 18089 consensus_queue.cc:1035] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcf48d96894746e08e1ca33b78bea683" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:55.106014 18083 consensus_queue.cc:1035] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "994ef811a0e34551a75344f6a17f7197" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:01:55.109787 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.108960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:48537)
0811 09:01:55.109358 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.109494 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221}
I20260811 09:01:55.110613 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.108109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:59851)
0811 09:01:55.108375 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.110321 (+ 1946us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 09:01:55.117535 18094 mvcc.cc:205] Tried to move back new op lower bound from 7317253796259368960 to 7317253795963920384. Current Snapshot: MvccSnapshot[applied={T|T < 7317253796259368960}]
I20260811 09:01:55.119215 18080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.118281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.67:34201)
0811 09:01:55.118757 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.118930 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309}
I20260811 09:01:55.120954 18081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.117687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.65:39041)
0811 09:01:55.120516 (+ 2829us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.120671 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2443}
I20260811 09:01:55.125029 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.123911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:51127)
0811 09:01:55.124368 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.124560 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 09:01:55.128383 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.126037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.9.40.66:45995)
0811 09:01:55.126295 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.126403 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 09:01:55.141990 17444 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:55.142352 17444 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:01:55.149911 17444 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 086f89fe0d4e44a1aa325df64a13e68e: Sending DeleteTablet for 3 replicas of tablet 7ba3dafba7df45338862f49cf1670676
I20260811 09:01:55.151289 17574 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba3dafba7df45338862f49cf1670676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:55 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:48752
I20260811 09:01:55.151640 17894 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba3dafba7df45338862f49cf1670676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:55 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:42834
I20260811 09:01:55.152137 17739 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ba3dafba7df45338862f49cf1670676 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:01:55 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:36704
I20260811 09:01:55.152599 18108 tablet_replica.cc:330] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: stopping tablet replica
I20260811 09:01:55.152999 18109 tablet_replica.cc:330] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: stopping tablet replica
I20260811 09:01:55.152913 18108 raft_consensus.cc:2229] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:55.153615 18108 raft_consensus.cc:2258] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:55.154608 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 17431
I20260811 09:01:55.157090 18109 raft_consensus.cc:2229] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:01:55.157789 18109 raft_consensus.cc:2258] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:55.158915 18109 ts_tablet_manager.cc:1905] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:55.159379 18108 ts_tablet_manager.cc:1905] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:55.164604 18109 ts_tablet_manager.cc:1918] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:55.164844 18109 log.cc:1196] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/7ba3dafba7df45338862f49cf1670676
I20260811 09:01:55.165371 18109 ts_tablet_manager.cc:1939] T 7ba3dafba7df45338862f49cf1670676 P 72af8e72ace1481ea8a2f10a379b4729: Deleting consensus metadata
I20260811 09:01:55.165591 18107 tablet_replica.cc:330] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: stopping tablet replica
I20260811 09:01:55.165879 18107 raft_consensus.cc:2229] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:01:55.166342 18107 raft_consensus.cc:2258] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:01:55.166878 17428 catalog_manager.cc:4841] TS 72af8e72ace1481ea8a2f10a379b4729 (127.9.40.66:46395): tablet 7ba3dafba7df45338862f49cf1670676 (table test-table [id=3050755a2b234d219a1d263558252ada]) successfully deleted
I20260811 09:01:55.168219 18108 ts_tablet_manager.cc:1918] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:55.168342 18108 log.cc:1196] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/7ba3dafba7df45338862f49cf1670676
I20260811 09:01:55.168885 18108 ts_tablet_manager.cc:1939] T 7ba3dafba7df45338862f49cf1670676 P 994ef811a0e34551a75344f6a17f7197: Deleting consensus metadata
I20260811 09:01:55.169519 18107 ts_tablet_manager.cc:1905] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:01:55.170321 17432 catalog_manager.cc:4841] TS 994ef811a0e34551a75344f6a17f7197 (127.9.40.67:36897): tablet 7ba3dafba7df45338862f49cf1670676 (table test-table [id=3050755a2b234d219a1d263558252ada]) successfully deleted
I20260811 09:01:55.177660 18107 ts_tablet_manager.cc:1918] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:01:55.177949 18107 log.cc:1196] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/7ba3dafba7df45338862f49cf1670676
I20260811 09:01:55.178525 18107 ts_tablet_manager.cc:1939] T 7ba3dafba7df45338862f49cf1670676 P bcf48d96894746e08e1ca33b78bea683: Deleting consensus metadata
I20260811 09:01:55.180032 17430 catalog_manager.cc:4841] TS bcf48d96894746e08e1ca33b78bea683 (127.9.40.65:42497): tablet 7ba3dafba7df45338862f49cf1670676 (table test-table [id=3050755a2b234d219a1d263558252ada]) successfully deleted
I20260811 09:01:55.184134 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.156191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.9.40.65:42497)
0811 09:01:55.156406 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.156412 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.167049 (+ 10637us) server_negotiation.cc:407] Connection header received
0811 09:01:55.167552 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.167558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.167660 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.167784 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:55.170899 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.171770 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.173237 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.173672 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.173825 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.176905 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:55.176925 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:55.176945 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:55.176989 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:55.180775 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.181654 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.181663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.181665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.181826 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.182246 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.182250 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.182252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.182670 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:55.183007 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.183450 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.183758 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:01:55.185002 17646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:55.287817 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 17431
I20260811 09:01:55.314599 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 17669
I20260811 09:01:55.334060 18069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.315643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.9.40.66:46395)
0811 09:01:55.315870 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.315878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.315937 (+ 59us) server_negotiation.cc:407] Connection header received
0811 09:01:55.316142 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.316149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.316246 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.316371 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:55.318120 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.319449 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.321234 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.321952 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.322104 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.324985 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:55.325010 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:55.325039 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:55.325097 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:55.329691 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.330836 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.330848 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.330852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.331085 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.332047 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.332053 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.332057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.332600 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:55.333009 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.333324 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.333623 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:01:55.334980 17805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:55.397150 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.396294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:35028)
0811 09:01:55.396841 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.396932 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":305,"threads_started":1}
I20260811 09:01:55.398586 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.398192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45611 (local address 127.0.0.1:35036)
0811 09:01:55.398362 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:55.398448 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:45611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:01:55.465332 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 17669
I20260811 09:01:55.492980 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 17824
I20260811 09:01:55.509756 18068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:55.493864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42848 (local address 127.9.40.67:36897)
0811 09:01:55.494065 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:55.494070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:55.494090 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:01:55.494405 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:55.494412 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:55.494525 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:55.494683 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:55.496331 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.497250 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.498673 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:55.499116 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:55.499246 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:55.502740 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:55.502762 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:55.502781 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:55.502834 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:55.506369 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.507372 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.507382 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.507385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.507578 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:55.507966 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:55.507969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:55.507971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:55.508421 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:55.508680 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:55.509141 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 09:01:55.509359 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:01:55.510500 17963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:55.641870 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 17824
I20260811 09:01:55.673329 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 17064
I20260811 09:01:55.674424 17259 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:55.799114 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 17064
I20260811 09:01:55.824898 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 17183
I20260811 09:01:55.826116 17482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:01:55.954269 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 17183
2026-08-11T09:01:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11131 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:01:56 dist-test-slave-cbsf krb5kdc[18121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:01:56 dist-test-slave-cbsf krb5kdc[18121](info): set up 2 sockets
Aug 11 09:01:56 dist-test-slave-cbsf krb5kdc[18121](info): commencing operation
krb5kdc: starting...
W20260811 09:01:57.178725 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:01:57 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:01:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
I20260811 09:01:57.474040 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38903
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:57.846640 18137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:57.846987 18137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:57.847083 18137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:57.854835 18137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:01:57.854979 18137 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:01:57.855021 18137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:57.855079 18137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:57.855140 18137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:01:57.855207 18137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:01:57.866345 18137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38903
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:57.867839 18137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:57.869282 18137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:57.883303 18144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:57.883277 18142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:57.883244 18146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:57.883975 18137 server_base.cc:1034] running on GCE node
I20260811 09:01:57.885119 18137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:57.886777 18137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:57.888051 18137 hybrid_clock.cc:648] HybridClock initialized: now 1786438917887989 us; error 44 us; skew 500 ppm
I20260811 09:01:57.888443 18137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:57 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438917, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:57.896464 18137 init.cc:377] Logged in from keytab as oryx/127.9.40.126@KRBTEST.COM (short username oryx)
I20260811 09:01:57.898998 18137 webserver.cc:460] Webserver started at http://127.9.40.126:34685/ using document root <none> and password file <none>
I20260811 09:01:57.899863 18137 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:57.899973 18137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:57.900310 18137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:57.904333 18137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "bc4746867fe34c4081a1d7af30cea312"
format_stamp: "Formatted at 2026-08-11 09:01:57 on dist-test-slave-cbsf"
server_key: "9300708d5442bdf4bc70c442ebdde602"
server_key_iv: "cb30950b5248d2fa39727d08734bd13e"
server_key_version: "encryptionkey@0"
I20260811 09:01:57.905400 18137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bc4746867fe34c4081a1d7af30cea312"
format_stamp: "Formatted at 2026-08-11 09:01:57 on dist-test-slave-cbsf"
server_key: "9300708d5442bdf4bc70c442ebdde602"
server_key_iv: "cb30950b5248d2fa39727d08734bd13e"
server_key_version: "encryptionkey@0"
I20260811 09:01:57.912813 18137 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 09:01:57.917343 18156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:57.919075 18137 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.004s
I20260811 09:01:57.919341 18137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "bc4746867fe34c4081a1d7af30cea312"
format_stamp: "Formatted at 2026-08-11 09:01:57 on dist-test-slave-cbsf"
server_key: "9300708d5442bdf4bc70c442ebdde602"
server_key_iv: "cb30950b5248d2fa39727d08734bd13e"
server_key_version: "encryptionkey@0"
I20260811 09:01:57.919627 18137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:57.949023 18137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:57.954176 18137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:57.954563 18137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:57.979753 18137 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38903
I20260811 09:01:57.979734 18227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38903 every 8 connection(s)
I20260811 09:01:57.981451 18137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:01:57.985551 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 18137
I20260811 09:01:57.985826 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:01:57.986315 18228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:01:57.986549 9377 external_mini_cluster.cc:1428] Setting key b92a5aa77e6897de965aee68c1f7cc28
I20260811 09:01:58.004583 18228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Bootstrap starting.
Aug 11 09:01:58 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:58.009203 18228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Neither blocks nor log segments found. Creating new log.
I20260811 09:01:58.010842 18228 log.cc:824] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Log is configured to *not* fsync() on all Append() calls
I20260811 09:01:58.015404 18228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: No bootstrap required, opened a new log
I20260811 09:01:58.017843 18231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:57.988904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.9.40.126:38903)
0811 09:01:57.989959 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:57.989980 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:57.990048 (+ 68us) server_negotiation.cc:407] Connection header received
0811 09:01:57.992566 (+ 2518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:57.992609 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:57.993344 (+ 735us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:57.993987 (+ 643us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:57.995783 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:57.998120 (+ 2337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:57.999748 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.001254 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.001512 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:58.006747 (+ 5235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:58.006762 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:58.006788 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:58.006859 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:58.011617 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.013093 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.013103 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.013114 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.013382 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.013893 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.013897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.013900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.014747 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:58.015129 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:58.015620 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 09:01:58.016210 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":418,"threads_started":1}
I20260811 09:01:58.025503 18228 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } }
I20260811 09:01:58.026058 18228 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:01:58.026186 18228 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc4746867fe34c4081a1d7af30cea312, State: Initialized, Role: FOLLOWER
I20260811 09:01:58.026947 18228 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } }
I20260811 09:01:58.027263 18228 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:01:58.027424 18228 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:01:58.027608 18228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:01:58.030293 18228 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } }
I20260811 09:01:58.031015 18228 leader_election.cc:302] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc4746867fe34c4081a1d7af30cea312; no voters:
I20260811 09:01:58.031878 18228 leader_election.cc:288] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:01:58.032064 18235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:01:58.032981 18235 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [term 1 LEADER]: Becoming Leader. State: Replica: bc4746867fe34c4081a1d7af30cea312, State: Running, Role: LEADER
I20260811 09:01:58.033746 18235 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } }
I20260811 09:01:58.034795 18228 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:01:58.038326 18236 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc4746867fe34c4081a1d7af30cea312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } } }
I20260811 09:01:58.039649 18236 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:58.038862 18239 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc4746867fe34c4081a1d7af30cea312. Latest consensus state: current_term: 1 leader_uuid: "bc4746867fe34c4081a1d7af30cea312" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc4746867fe34c4081a1d7af30cea312" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38903 } } }
I20260811 09:01:58.040598 18239 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312 [sys.catalog]: This master's current role is: LEADER
I20260811 09:01:58.046224 18247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:01:58.053534 18247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:01:58.065959 18247 catalog_manager.cc:1347] Generated new cluster ID: 42f9d39f0df34ea98c199477bd72c762
I20260811 09:01:58.066188 18247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:01:58.083433 18247 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:01:58.085018 18247 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:01:58.099012 18247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Generated new TSK 0
I20260811 09:01:58.100251 18247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
I20260811 09:01:58.196223 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38903
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:58.706990 18262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:58.707341 18262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:58.707464 18262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:58.715531 18262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:58.715682 18262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:58.715775 18262 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:58.715860 18262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:01:58.726958 18262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:58.728396 18262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:58.729908 18262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:58.746095 18269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:58.746501 18271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:58.746095 18268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:58.747054 18262 server_base.cc:1034] running on GCE node
I20260811 09:01:58.748109 18262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:58.749430 18262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:58.750748 18262 hybrid_clock.cc:648] HybridClock initialized: now 1786438918750671 us; error 54 us; skew 500 ppm
I20260811 09:01:58.751132 18262 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:58 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438918, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:58.755112 18262 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260811 09:01:58.757218 18262 webserver.cc:460] Webserver started at http://127.9.40.65:43527/ using document root <none> and password file <none>
I20260811 09:01:58.758155 18262 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:58.758316 18262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:58.758709 18262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:58.762368 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "e60ba19686914e048874af71180dd32d"
format_stamp: "Formatted at 2026-08-11 09:01:58 on dist-test-slave-cbsf"
server_key: "3fea4aacaffd5dc84eb5928e5a7ece87"
server_key_iv: "4d27dd6cfabebccdae0c6c8c8febb31a"
server_key_version: "encryptionkey@0"
I20260811 09:01:58.763418 18262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "e60ba19686914e048874af71180dd32d"
format_stamp: "Formatted at 2026-08-11 09:01:58 on dist-test-slave-cbsf"
server_key: "3fea4aacaffd5dc84eb5928e5a7ece87"
server_key_iv: "4d27dd6cfabebccdae0c6c8c8febb31a"
server_key_version: "encryptionkey@0"
I20260811 09:01:58.770107 18262 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.006s
I20260811 09:01:58.774598 18279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:58.776201 18262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:01:58.776440 18262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "e60ba19686914e048874af71180dd32d"
format_stamp: "Formatted at 2026-08-11 09:01:58 on dist-test-slave-cbsf"
server_key: "3fea4aacaffd5dc84eb5928e5a7ece87"
server_key_iv: "4d27dd6cfabebccdae0c6c8c8febb31a"
server_key_version: "encryptionkey@0"
I20260811 09:01:58.776759 18262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:58.791106 18262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:58.796236 18262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:58.796586 18262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:58.801015 18262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:58.801251 18262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:58.801530 18262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:58.801652 18262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:58 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438918, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:58.833501 18287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:58.801544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53401 (local address 127.9.40.126:38903)
0811 09:01:58.808974 (+ 7430us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:58.808985 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:58.809013 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:01:58.810821 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:58.810827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:58.810940 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:58.811087 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:58.814139 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.815543 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.818753 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.819618 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.819819 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:58.826047 (+ 6228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:58.826054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:58.826057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:58.826103 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:58.829279 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.830617 (+ 1338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.830623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.830625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.830785 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.831373 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.831377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.831379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.831706 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:58.831889 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:58.832801 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 09:01:58.833104 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7266,"thread_start_us":7118,"threads_started":1}
I20260811 09:01:58.835297 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:58.802225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.65:53401)
0811 09:01:58.807940 (+ 5715us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:58.808025 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:58.810410 (+ 2385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:58.811411 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:58.811421 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:58.812614 (+ 1193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:58.813800 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:58.813830 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.815846 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.815853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:58.818497 (+ 2644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:58.818520 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.820093 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.820113 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:58.820399 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:58.821441 (+ 1042us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:58.821469 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:58.825545 (+ 4076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:58.829495 (+ 3950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:58.829507 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:58.829548 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:58.830419 (+ 871us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:58.830925 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:58.830930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:58.830934 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:58.831200 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:58.831861 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:58.831871 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:58.832309 (+ 438us) client_negotiation.cc:769] Sending connection context
0811 09:01:58.832622 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 09:01:58.833304 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5333,"thread_start_us":4972,"threads_started":1}
I20260811 09:01:58.858954 18287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:58.843327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56815 (local address 127.9.40.126:38903)
0811 09:01:58.843575 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:58.843582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:58.843789 (+ 207us) server_negotiation.cc:407] Connection header received
0811 09:01:58.844031 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:58.844036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:58.844107 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:58.844220 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:58.845742 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.846893 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.849321 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.849797 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.849959 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:58.852112 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:58.852130 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:58.852133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:58.852172 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:58.855274 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.856412 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.856419 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.856423 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.856590 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:58.857069 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:58.857077 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:58.857081 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:58.857568 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:58.857761 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:58.858250 (+ 489us) server_negotiation.cc:299] Negotiation successful
0811 09:01:58.858542 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 09:01:58.859270 18371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:58.842869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.65:56815)
0811 09:01:58.843669 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:58.843692 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:58.843880 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:58.844254 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:58.844259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:58.844915 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:58.845444 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:58.845469 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.847062 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.847069 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:58.849073 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:58.849104 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:58.850038 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:58.850045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:58.850225 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:58.850947 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:58.850967 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:58.851842 (+ 875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:58.855510 (+ 3668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:58.855514 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:58.855522 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:58.856241 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:58.856768 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:58.856773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:58.856775 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:58.856916 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:58.857710 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:58.857716 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:58.857940 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 09:01:58.858129 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 09:01:58.858623 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":407,"threads_started":1}
I20260811 09:01:58.865204 18262 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45633
I20260811 09:01:58.865233 18415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45633 every 8 connection(s)
I20260811 09:01:58.866600 18262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:01:58.869128 18416 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38903
I20260811 09:01:58.869436 18416 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:58.870225 18416 heartbeater.cc:511] Master 127.9.40.126:38903 requested a full tablet report, sending...
I20260811 09:01:58.873391 18175 ts_manager.cc:194] Registered new tserver with Master: e60ba19686914e048874af71180dd32d (127.9.40.65:45633)
I20260811 09:01:58.874197 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 18262
I20260811 09:01:58.874465 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:01:58.875422 9377 external_mini_cluster.cc:1428] Setting key 15c0608685d777e2649fb8a47054e4ad
I20260811 09:01:58.876041 18175 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:53401
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
I20260811 09:01:58.943892 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38903
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:01:59.338449 18420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:59.338858 18420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:59.339042 18420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:59.347266 18420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:59.347476 18420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:59.347636 18420 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:59.347733 18420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:01:59.358906 18420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:59.360559 18420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:59.362133 18420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:59.377764 18426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:59.377830 18429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:59.377856 18427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:59.378931 18420 server_base.cc:1034] running on GCE node
I20260811 09:01:59.379642 18420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:59.380851 18420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:59.382099 18420 hybrid_clock.cc:648] HybridClock initialized: now 1786438919382022 us; error 44 us; skew 500 ppm
I20260811 09:01:59.382423 18420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:59 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438919, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:01:59.386168 18420 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260811 09:01:59.387969 18420 webserver.cc:460] Webserver started at http://127.9.40.66:45823/ using document root <none> and password file <none>
I20260811 09:01:59.388847 18420 fs_manager.cc:359] Metadata directory not provided
I20260811 09:01:59.388978 18420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:01:59.389370 18420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:01:59.392791 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "fd2176f081ce4f588a56a3f745eab32a"
format_stamp: "Formatted at 2026-08-11 09:01:59 on dist-test-slave-cbsf"
server_key: "c373b0187109430d1a162f4b2a49c9ad"
server_key_iv: "0cccddd1f3ce462b451e3940d6b93a16"
server_key_version: "encryptionkey@0"
I20260811 09:01:59.393826 18420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "fd2176f081ce4f588a56a3f745eab32a"
format_stamp: "Formatted at 2026-08-11 09:01:59 on dist-test-slave-cbsf"
server_key: "c373b0187109430d1a162f4b2a49c9ad"
server_key_iv: "0cccddd1f3ce462b451e3940d6b93a16"
server_key_version: "encryptionkey@0"
I20260811 09:01:59.400166 18420 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:01:59.404390 18436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:59.406189 18420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:01:59.406430 18420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "fd2176f081ce4f588a56a3f745eab32a"
format_stamp: "Formatted at 2026-08-11 09:01:59 on dist-test-slave-cbsf"
server_key: "c373b0187109430d1a162f4b2a49c9ad"
server_key_iv: "0cccddd1f3ce462b451e3940d6b93a16"
server_key_version: "encryptionkey@0"
I20260811 09:01:59.406716 18420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:01:59.424045 18420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:01:59.429520 18420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:01:59.429881 18420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:01:59.433192 18420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:01:59.433336 18420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:01:59.433514 18420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:01:59.433604 18420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:01:59 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438919, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:01:59.463645 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:59.434114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50821 (local address 127.9.40.126:38903)
0811 09:01:59.434694 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:59.434700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:59.436517 (+ 1817us) server_negotiation.cc:407] Connection header received
0811 09:01:59.440278 (+ 3761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:59.440292 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:59.440386 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:59.440535 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:59.443468 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.444629 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.447517 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.448592 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.448839 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:59.454000 (+ 5161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:59.454013 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:59.454023 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:59.454094 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:59.458212 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:59.459760 (+ 1548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:59.459767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:59.459770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:59.460031 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:59.460965 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:59.460974 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:59.460979 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:59.461412 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:59.461622 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:59.463067 (+ 1445us) server_negotiation.cc:299] Negotiation successful
0811 09:01:59.463337 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":260,"threads_started":1}
I20260811 09:01:59.466012 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:59.435001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.66:50821)
0811 09:01:59.436251 (+ 1250us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:59.436325 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:59.439710 (+ 3385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:59.440974 (+ 1264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:59.440994 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:59.441990 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:59.443187 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:59.443214 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.444891 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.444896 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:59.447252 (+ 2356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:59.447283 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.448944 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.448949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:59.449191 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:59.450184 (+ 993us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:59.450216 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:59.453546 (+ 3330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:59.458458 (+ 4912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:59.458476 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:59.458521 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:59.459547 (+ 1026us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:59.460238 (+ 691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:59.460273 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:59.460279 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:59.460761 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:59.461626 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:59.461642 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:59.462395 (+ 753us) client_negotiation.cc:769] Sending connection context
0811 09:01:59.462882 (+ 487us) client_negotiation.cc:240] Negotiation successful
0811 09:01:59.463774 (+ 892us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":449,"threads_started":1}
I20260811 09:01:59.489956 18420 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:44663
I20260811 09:01:59.490000 18573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:44663 every 8 connection(s)
I20260811 09:01:59.491433 18420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:01:59.493585 18446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:59.474218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35813 (local address 127.9.40.126:38903)
0811 09:01:59.474498 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 09:01:59.474508 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:01:59.475486 (+ 978us) server_negotiation.cc:407] Connection header received
0811 09:01:59.475645 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:01:59.475653 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:01:59.475760 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:01:59.475904 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:01:59.477557 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.478901 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.481654 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.482365 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.482525 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:59.485982 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:01:59.485993 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:01:59.486000 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:01:59.486054 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:01:59.489679 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:59.490705 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:59.490712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:59.490716 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:59.490872 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:01:59.491299 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:01:59.491304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:01:59.491307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:01:59.491741 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:01:59.491914 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 09:01:59.492385 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 09:01:59.492822 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:01:59.494642 18574 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38903
I20260811 09:01:59.494653 18529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:01:59.474198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.66:35813)
0811 09:01:59.475324 (+ 1126us) negotiation.cc:107] Waiting for socket to connect
0811 09:01:59.475347 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:01:59.475514 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:01:59.475973 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:01:59.475980 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:01:59.476793 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:01:59.477287 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:59.477302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.479141 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.479150 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:59.481406 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:01:59.481432 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:01:59.482640 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:01:59.482648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:01:59.482950 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:01:59.483804 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:01:59.483847 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:01:59.485666 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:01:59.489858 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:59.489864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:59.489868 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:59.490549 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:59.491008 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:01:59.491014 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:01:59.491017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:01:59.491170 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:01:59.491871 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:01:59.491876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:01:59.492091 (+ 215us) client_negotiation.cc:769] Sending connection context
0811 09:01:59.492276 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 09:01:59.492670 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"thread_start_us":782,"threads_started":1}
I20260811 09:01:59.495042 18574 heartbeater.cc:464] Registering TS with master...
I20260811 09:01:59.496286 18574 heartbeater.cc:511] Master 127.9.40.126:38903 requested a full tablet report, sending...
I20260811 09:01:59.498484 18174 ts_manager.cc:194] Registered new tserver with Master: fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663)
I20260811 09:01:59.498580 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 18420
I20260811 09:01:59.498775 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:01:59.499331 9377 external_mini_cluster.cc:1428] Setting key e9599a325b236927303c05610063e387
I20260811 09:01:59.499804 18174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:50821
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260811 09:01:59.564174 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38903
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:01:59.880025 18416 heartbeater.cc:503] Master 127.9.40.126:38903 was elected leader, sending a full tablet report...
W20260811 09:01:59.951575 18578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:01:59.951910 18578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:01:59.952081 18578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:01:59.959719 18578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:01:59.959874 18578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:01:59.960054 18578 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:01:59.960196 18578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:01:59.971026 18578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:38903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:01:59.972685 18578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:01:59.974257 18578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:01:59.991834 18583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:59.991791 18587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:01:59.991757 18584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:01:59.993058 18578 server_base.cc:1034] running on GCE node
I20260811 09:01:59.993907 18578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:01:59.995448 18578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:01:59.996786 18578 hybrid_clock.cc:648] HybridClock initialized: now 1786438919996654 us; error 58 us; skew 500 ppm
I20260811 09:01:59.997206 18578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:01:59 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438919, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:00.001644 18578 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260811 09:02:00.003521 18578 webserver.cc:460] Webserver started at http://127.9.40.67:45379/ using document root <none> and password file <none>
I20260811 09:02:00.004376 18578 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:00.004536 18578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:00.004980 18578 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:00.008590 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "a2c278560eaf476da7697747d587fdb1"
format_stamp: "Formatted at 2026-08-11 09:02:00 on dist-test-slave-cbsf"
server_key: "5aa0cfac9b37abd884d7f133af60cad0"
server_key_iv: "1d57c8cbbb959bdcebe290bfd37a3014"
server_key_version: "encryptionkey@0"
I20260811 09:02:00.009676 18578 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "a2c278560eaf476da7697747d587fdb1"
format_stamp: "Formatted at 2026-08-11 09:02:00 on dist-test-slave-cbsf"
server_key: "5aa0cfac9b37abd884d7f133af60cad0"
server_key_iv: "1d57c8cbbb959bdcebe290bfd37a3014"
server_key_version: "encryptionkey@0"
I20260811 09:02:00.016291 18578 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:02:00.020665 18597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:00.022378 18578 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:00.022619 18578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "a2c278560eaf476da7697747d587fdb1"
format_stamp: "Formatted at 2026-08-11 09:02:00 on dist-test-slave-cbsf"
server_key: "5aa0cfac9b37abd884d7f133af60cad0"
server_key_iv: "1d57c8cbbb959bdcebe290bfd37a3014"
server_key_version: "encryptionkey@0"
I20260811 09:02:00.022890 18578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:00.043960 18578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:00.049455 18578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:00.049793 18578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:00.053169 18578 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:00.053313 18578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:00.053457 18578 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:00.053532 18578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438919, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:02:00.083701 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.054112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58875 (local address 127.9.40.126:38903)
0811 09:02:00.054606 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.054611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.056406 (+ 1795us) server_negotiation.cc:407] Connection header received
0811 09:02:00.058700 (+ 2294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.058708 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.058817 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.058949 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:00.060772 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.062195 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.065392 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.066135 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.066327 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.073378 (+ 7051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:00.073390 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:00.073395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:00.073458 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:00.078136 (+ 4678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.079863 (+ 1727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.079872 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.079876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.080078 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.081046 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.081051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.081053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.081552 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:00.081742 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.083124 (+ 1382us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.083375 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":255,"threads_started":1}
I20260811 09:02:00.086148 18609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.054822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.67:58875)
0811 09:02:00.055956 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.056013 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.058244 (+ 2231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.059264 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.059274 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.059881 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:00.060519 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.060538 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.062436 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.062445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.065142 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.065176 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.066567 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.066579 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.066996 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.068068 (+ 1072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:00.068120 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:00.072827 (+ 4707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:00.078451 (+ 5624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.078466 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.078501 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.079610 (+ 1109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.080244 (+ 634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.080252 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.080259 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.080864 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.081742 (+ 878us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:00.081757 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:00.082529 (+ 772us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.082993 (+ 464us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.083844 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":493,"threads_started":1}
I20260811 09:02:00.107626 18578 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:46213
I20260811 09:02:00.107627 18731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:46213 every 8 connection(s)
I20260811 09:02:00.109329 18578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:02:00.112293 18732 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38903
I20260811 09:02:00.112623 18732 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:00.113627 18732 heartbeater.cc:511] Master 127.9.40.126:38903 requested a full tablet report, sending...
I20260811 09:02:00.115367 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.096347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.9.40.67:59065)
0811 09:02:00.097384 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.097414 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.097718 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.098140 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.098145 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.098910 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:00.099255 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.099267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.101027 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.101037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.103445 (+ 2408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.103480 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.105296 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.105303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.105485 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.106346 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:00.106375 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:00.107904 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:00.112427 (+ 4523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.112433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.112437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.113114 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.113546 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.113551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.113554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.113729 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.114293 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:00.114299 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:00.114518 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.114659 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.115040 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":589,"threads_started":1}
I20260811 09:02:00.115427 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.096219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59065 (local address 127.9.40.126:38903)
0811 09:02:00.096459 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.096467 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.097544 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 09:02:00.097886 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.097892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.097981 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.098105 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:00.099438 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.100751 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.103798 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.104760 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.104948 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.108277 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:00.108284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:00.108287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:00.108329 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:00.112266 (+ 3937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.113251 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.113256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.113259 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.113409 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.113840 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.113844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.113846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.114146 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:00.114263 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.114810 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.115053 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:02:00.115443 18174 ts_manager.cc:194] Registered new tserver with Master: a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:00.116665 18174 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:58875
I20260811 09:02:00.117228 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 18578
I20260811 09:02:00.117443 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:02:00.118013 9377 external_mini_cluster.cc:1428] Setting key 708ae586b11d81f2aefddb19854ae0fa
I20260811 09:02:00.121376 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438920, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:02:00.191265 18608 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:00.178666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.9.40.126:38903)
0811 09:02:00.178869 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.178875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.179163 (+ 288us) server_negotiation.cc:407] Connection header received
0811 09:02:00.179330 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.179335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.179433 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.179539 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:00.181226 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.181969 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.183565 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.183991 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.184101 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.190842 (+ 6741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49712: BlockingRecv error: recv got EOF from 127.0.0.1:49712 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438920, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:02:00.210691 18608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.195208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.9.40.126:38903)
0811 09:02:00.195477 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.195484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.195706 (+ 222us) server_negotiation.cc:407] Connection header received
0811 09:02:00.195863 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.195867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.195937 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.196028 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:00.197473 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.198268 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.199700 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.200179 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.200289 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.204806 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:00.204813 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:00.204817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:00.204865 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:00.207710 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.208572 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.208578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.208581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.208822 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.209257 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.209261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.209263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.209582 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:00.209786 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.210174 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.210391 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 09:02:00.215384 18174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:02:00.223160 18174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:00.266170 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.239000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34666 (local address 127.9.40.66:44663)
0811 09:02:00.247494 (+ 8494us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.247511 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.251978 (+ 4467us) server_negotiation.cc:407] Connection header received
0811 09:02:00.252119 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.252132 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.252542 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.252849 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.254136 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.256389 (+ 2253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.262821 (+ 6432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.264427 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.264552 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.264854 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.265604 (+ 750us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.265819 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8207,"thread_start_us":7668,"threads_started":1}
I20260811 09:02:00.266263 18754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.238456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46213 (local address 127.0.0.1:49014)
0811 09:02:00.248036 (+ 9580us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.248119 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.248452 (+ 333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.250703 (+ 2251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.250710 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.250743 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.251121 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.251132 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.255725 (+ 4593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.255736 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.263121 (+ 7385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.263136 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.265177 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.265184 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.265425 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.265460 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.265586 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.265841 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9057,"mutex_wait_us":795,"thread_start_us":9092,"threads_started":1}
I20260811 09:02:00.266224 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.238840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.9.40.67:46213)
0811 09:02:00.249715 (+ 10875us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.249729 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.249779 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:02:00.249922 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.249935 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.250343 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.250665 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.251493 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.255372 (+ 3879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.263380 (+ 8008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.264941 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.265134 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.265387 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.265636 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.265818 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10615,"thread_start_us":10359,"threads_started":1}
I20260811 09:02:00.267062 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.238762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44663 (local address 127.0.0.1:34666)
0811 09:02:00.251646 (+ 12884us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.251674 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.251802 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.253165 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.253175 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.253217 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.253864 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.253881 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.256725 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.256735 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.262541 (+ 5806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.262561 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.264933 (+ 2372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.264939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.265186 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.265281 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.266463 (+ 1182us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.266625 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12734,"mutex_wait_us":345,"thread_start_us":12524,"threads_started":1}
I20260811 09:02:00.267992 18758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.251847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45633 (local address 127.0.0.1:49488)
0811 09:02:00.259503 (+ 7656us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.259524 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.259756 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.260532 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.260537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.260568 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.261161 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.261173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.262667 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.262673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.264539 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.264555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.266888 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.266895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.267128 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.267347 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.267463 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.267611 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7469,"mutex_wait_us":1,"thread_start_us":7113,"threads_started":1}
I20260811 09:02:00.268850 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.252491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49488 (local address 127.9.40.65:45633)
0811 09:02:00.257712 (+ 5221us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.257720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.259713 (+ 1993us) server_negotiation.cc:407] Connection header received
0811 09:02:00.260041 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.260050 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.260303 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.260551 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.261362 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.262508 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.264778 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.266369 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.266516 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.266729 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.267531 (+ 802us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.267735 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5020,"thread_start_us":4867,"threads_started":1}
I20260811 09:02:00.271246 18655 tablet_service.cc:1467] Processing CreateTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef (DEFAULT_TABLE table=test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.271245 18495 tablet_service.cc:1467] Processing CreateTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef (DEFAULT_TABLE table=test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.273066 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cad8a6edec4c3c84fc3707ddd749ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.273201 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cad8a6edec4c3c84fc3707ddd749ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.276932 18337 tablet_service.cc:1467] Processing CreateTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef (DEFAULT_TABLE table=test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.278402 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43cad8a6edec4c3c84fc3707ddd749ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.290549 18761 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Bootstrap starting.
I20260811 09:02:00.292078 18760 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Bootstrap starting.
I20260811 09:02:00.292996 18763 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Bootstrap starting.
I20260811 09:02:00.296370 18763 tablet_bootstrap.cc:654] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.296372 18760 tablet_bootstrap.cc:654] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.298012 18760 log.cc:824] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:00.298274 18763 log.cc:824] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:00.300964 18761 tablet_bootstrap.cc:654] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.302274 18763 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: No bootstrap required, opened a new log
I20260811 09:02:00.302652 18763 ts_tablet_manager.cc:1397] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 09:02:00.302750 18761 log.cc:824] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:00.305657 18760 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: No bootstrap required, opened a new log
I20260811 09:02:00.306015 18760 ts_tablet_manager.cc:1397] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260811 09:02:00.307883 18761 tablet_bootstrap.cc:492] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: No bootstrap required, opened a new log
I20260811 09:02:00.308198 18761 ts_tablet_manager.cc:1397] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.003s
I20260811 09:02:00.314807 18763 raft_consensus.cc:348] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.315405 18763 raft_consensus.cc:374] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.315531 18763 raft_consensus.cc:729] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60ba19686914e048874af71180dd32d, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.316313 18763 consensus_queue.cc:260] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.316452 18761 raft_consensus.cc:348] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.317001 18761 raft_consensus.cc:374] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.317111 18761 raft_consensus.cc:729] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.317957 18761 consensus_queue.cc:260] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.318053 18760 raft_consensus.cc:348] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.318660 18760 raft_consensus.cc:374] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.318789 18760 raft_consensus.cc:729] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2176f081ce4f588a56a3f745eab32a, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.319687 18760 consensus_queue.cc:260] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.319983 18732 heartbeater.cc:503] Master 127.9.40.126:38903 was elected leader, sending a full tablet report...
I20260811 09:02:00.321913 18763 ts_tablet_manager.cc:1428] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260811 09:02:00.322948 18760 ts_tablet_manager.cc:1428] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260811 09:02:00.323541 18574 heartbeater.cc:503] Master 127.9.40.126:38903 was elected leader, sending a full tablet report...
I20260811 09:02:00.324473 18761 ts_tablet_manager.cc:1428] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Time spent starting tablet: real 0.016s user 0.009s sys 0.004s
I20260811 09:02:00.347414 18769 raft_consensus.cc:482] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:00.347805 18769 raft_consensus.cc:504] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.349752 18769 leader_election.cc:288] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), e60ba19686914e048874af71180dd32d (127.9.40.65:45633)
I20260811 09:02:00.357438 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.350249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49115 (local address 127.9.40.66:44663)
0811 09:02:00.350499 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.350509 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.350538 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:02:00.350791 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.350797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.350892 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.351003 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.351636 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.352857 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.355141 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.356423 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.356599 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.356760 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.357094 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.357227 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:02:00.357545 18609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.350241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44663 (local address 127.9.40.67:49115)
0811 09:02:00.350471 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.350489 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.350626 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.351054 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.351062 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.351135 (+ 73us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.351454 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.351470 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.353004 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.353010 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.354949 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.354963 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.356655 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.356660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.356899 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.356939 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.357050 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.357234 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":1}
I20260811 09:02:00.358750 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cad8a6edec4c3c84fc3707ddd749ef" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a" is_pre_election: true
I20260811 09:02:00.358978 18759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.351097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49429 (local address 127.9.40.65:45633)
0811 09:02:00.351317 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.351326 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.351813 (+ 487us) server_negotiation.cc:407] Connection header received
0811 09:02:00.351963 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.351968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.352041 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.352121 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.352810 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.353897 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.356646 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.357987 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.358101 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.358192 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.358562 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.358703 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:02:00.359212 18771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.350764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45633 (local address 127.9.40.67:49429)
0811 09:02:00.351697 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.351719 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.351856 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.352163 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.352168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.352193 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.352559 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.352570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.354072 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.354077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.356430 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.356449 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.358224 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.358232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.358383 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.358409 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.358529 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.358703 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"mutex_wait_us":294,"thread_start_us":559,"threads_started":1}
I20260811 09:02:00.359386 18517 raft_consensus.cc:2454] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 0.
I20260811 09:02:00.360160 18600 leader_election.cc:302] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, fd2176f081ce4f588a56a3f745eab32a; no voters:
I20260811 09:02:00.360276 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cad8a6edec4c3c84fc3707ddd749ef" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60ba19686914e048874af71180dd32d" is_pre_election: true
I20260811 09:02:00.360622 18769 raft_consensus.cc:2781] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:00.360842 18769 raft_consensus.cc:482] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:00.360967 18769 raft_consensus.cc:3037] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.360989 18359 raft_consensus.cc:2454] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 0.
W20260811 09:02:00.361516 18733 tablet.cc:2369] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:02:00.363886 18769 raft_consensus.cc:504] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.364498 18769 leader_election.cc:288] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 election: Requested vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), e60ba19686914e048874af71180dd32d (127.9.40.65:45633)
I20260811 09:02:00.365235 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cad8a6edec4c3c84fc3707ddd749ef" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a"
I20260811 09:02:00.365310 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43cad8a6edec4c3c84fc3707ddd749ef" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60ba19686914e048874af71180dd32d"
I20260811 09:02:00.365475 18517 raft_consensus.cc:3037] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.365619 18359 raft_consensus.cc:3037] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.368040 18517 raft_consensus.cc:2454] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 1.
I20260811 09:02:00.368034 18359 raft_consensus.cc:2454] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 1.
I20260811 09:02:00.368660 18600 leader_election.cc:302] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, fd2176f081ce4f588a56a3f745eab32a; no voters:
I20260811 09:02:00.369135 18769 raft_consensus.cc:2781] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 09:02:00.369503 18417 tablet.cc:2369] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:02:00.369763 18769 raft_consensus.cc:686] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 1 LEADER]: Becoming Leader. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Running, Role: LEADER
I20260811 09:02:00.370682 18769 consensus_queue.cc:237] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } }
I20260811 09:02:00.375646 18175 catalog_manager.cc:5495] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2c278560eaf476da7697747d587fdb1 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } health_report { overall_health: UNKNOWN } } }
I20260811 09:02:00.423808 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.418056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50710 (local address 127.9.40.67:46213)
0811 09:02:00.418256 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.418261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.418281 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:02:00.418442 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.418447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.418531 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.418689 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:00.419016 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.419756 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.421301 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.421764 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.421876 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.422429 (+ 553us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:00.423069 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:00.423211 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.423367 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.423457 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:02:00.433521 18359 raft_consensus.cc:1261] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Refusing update from remote peer a2c278560eaf476da7697747d587fdb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:00.433521 18517 raft_consensus.cc:1261] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Refusing update from remote peer a2c278560eaf476da7697747d587fdb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:00.434666 18769 consensus_queue.cc:1035] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:00.435101 18772 consensus_queue.cc:1035] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:00.445636 18775 mvcc.cc:205] Tried to move back new op lower bound from 7317253818083409920 to 7317253817842356224. Current Snapshot: MvccSnapshot[applied={T|T < 7317253818083409920}]
I20260811 09:02:00.447832 18777 mvcc.cc:205] Tried to move back new op lower bound from 7317253818083409920 to 7317253817842356224. Current Snapshot: MvccSnapshot[applied={T|T < 7317253818083409920}]
I20260811 09:02:00.464737 18175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:00.465121 18175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:00.470551 18175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Sending DeleteTablet for 3 replicas of tablet 43cad8a6edec4c3c84fc3707ddd749ef
I20260811 09:02:00.472141 18495 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:00.473067 18337 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49488
I20260811 09:02:00.473299 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49014
I20260811 09:02:00.473847 18494 tablet_service.cc:1514] Processing DeleteTablet for tablet 43cad8a6edec4c3c84fc3707ddd749ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:00.474152 18786 tablet_replica.cc:330] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: stopping tablet replica
I20260811 09:02:00.474680 18786 raft_consensus.cc:2229] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:00.475452 18786 raft_consensus.cc:2258] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:00.475569 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:02:00.477226 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:00.478502 18786 ts_tablet_manager.cc:1905] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:00.486398 18785 tablet_replica.cc:330] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: stopping tablet replica
I20260811 09:02:00.486997 18785 raft_consensus.cc:2229] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:00.487797 18785 raft_consensus.cc:2258] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:00.489583 18495 tablet_service.cc:1467] Processing CreateTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (DEFAULT_TABLE table=test-table [id=f9536c8c9cda4f5684636ea05fc79b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.489583 18337 tablet_service.cc:1467] Processing CreateTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (DEFAULT_TABLE table=test-table [id=f9536c8c9cda4f5684636ea05fc79b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.490113 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2859ce7a2246a0a7cbaea5140e8ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.490118 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2859ce7a2246a0a7cbaea5140e8ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.498894 18760 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Bootstrap starting.
I20260811 09:02:00.499976 18655 tablet_service.cc:1467] Processing CreateTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (DEFAULT_TABLE table=test-table [id=f9536c8c9cda4f5684636ea05fc79b76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:00.500501 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc2859ce7a2246a0a7cbaea5140e8ee5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.500977 18760 tablet_bootstrap.cc:654] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.504835 18787 tablet_replica.cc:330] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: stopping tablet replica
I20260811 09:02:00.505213 18763 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Bootstrap starting.
I20260811 09:02:00.505688 18787 raft_consensus.cc:2229] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:02:00.506469 18787 raft_consensus.cc:2258] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:00.506719 18785 ts_tablet_manager.cc:1905] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:00.507089 18763 tablet_bootstrap.cc:654] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.509119 18787 ts_tablet_manager.cc:1905] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:00.510725 18760 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: No bootstrap required, opened a new log
I20260811 09:02:00.510943 18760 ts_tablet_manager.cc:1397] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.006s
I20260811 09:02:00.511998 18760 raft_consensus.cc:348] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.512326 18760 raft_consensus.cc:374] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.512468 18760 raft_consensus.cc:729] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2176f081ce4f588a56a3f745eab32a, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.512925 18760 consensus_queue.cc:260] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.513576 18760 ts_tablet_manager.cc:1428] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 09:02:00.519268 18763 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: No bootstrap required, opened a new log
I20260811 09:02:00.519572 18763 ts_tablet_manager.cc:1397] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.000s
I20260811 09:02:00.520610 18787 ts_tablet_manager.cc:1918] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:00.520958 18787 log.cc:1196] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/43cad8a6edec4c3c84fc3707ddd749ef
I20260811 09:02:00.521626 18787 ts_tablet_manager.cc:1939] T 43cad8a6edec4c3c84fc3707ddd749ef P a2c278560eaf476da7697747d587fdb1: Deleting consensus metadata
I20260811 09:02:00.521500 18763 raft_consensus.cc:348] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.523813 18159 catalog_manager.cc:4841] TS a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213): tablet 43cad8a6edec4c3c84fc3707ddd749ef (table test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]) successfully deleted
I20260811 09:02:00.524161 18763 raft_consensus.cc:374] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.524653 18785 ts_tablet_manager.cc:1918] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:00.524936 18785 log.cc:1196] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/43cad8a6edec4c3c84fc3707ddd749ef
I20260811 09:02:00.525002 18763 raft_consensus.cc:729] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60ba19686914e048874af71180dd32d, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.525664 18785 ts_tablet_manager.cc:1939] T 43cad8a6edec4c3c84fc3707ddd749ef P fd2176f081ce4f588a56a3f745eab32a: Deleting consensus metadata
I20260811 09:02:00.525609 18763 consensus_queue.cc:260] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.527832 18786 ts_tablet_manager.cc:1918] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:00.527951 18158 catalog_manager.cc:4841] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): tablet 43cad8a6edec4c3c84fc3707ddd749ef (table test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]) successfully deleted
W20260811 09:02:00.529248 18158 catalog_manager.cc:4808] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): delete failed for tablet 43cad8a6edec4c3c84fc3707ddd749ef because the tablet was not found. No further retry: Not found: Tablet not found: 43cad8a6edec4c3c84fc3707ddd749ef
I20260811 09:02:00.529477 18763 ts_tablet_manager.cc:1428] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Time spent starting tablet: real 0.009s user 0.003s sys 0.000s
I20260811 09:02:00.529838 18786 log.cc:1196] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/43cad8a6edec4c3c84fc3707ddd749ef
I20260811 09:02:00.530575 18786 ts_tablet_manager.cc:1939] T 43cad8a6edec4c3c84fc3707ddd749ef P e60ba19686914e048874af71180dd32d: Deleting consensus metadata
I20260811 09:02:00.532758 18158 catalog_manager.cc:4841] TS e60ba19686914e048874af71180dd32d (127.9.40.65:45633): tablet 43cad8a6edec4c3c84fc3707ddd749ef (table test-table [id=0a7c8b4d7a1e42a7ae5eee837020d6a8]) successfully deleted
I20260811 09:02:00.533488 18761 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Bootstrap starting.
I20260811 09:02:00.534983 18761 tablet_bootstrap.cc:654] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.539132 18761 tablet_bootstrap.cc:492] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: No bootstrap required, opened a new log
I20260811 09:02:00.539354 18761 ts_tablet_manager.cc:1397] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 09:02:00.540153 18761 raft_consensus.cc:348] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.540356 18761 raft_consensus.cc:374] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.540483 18761 raft_consensus.cc:729] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.540854 18761 consensus_queue.cc:260] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.541464 18761 ts_tablet_manager.cc:1428] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:02:00.739728 18768 raft_consensus.cc:482] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:00.740044 18768 raft_consensus.cc:504] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.741968 18768 leader_election.cc:288] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
W20260811 09:02:00.748557 18575 tablet.cc:2369] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:02:00.752363 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.742405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46213 (local address 127.9.40.65:45965)
0811 09:02:00.743650 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.743674 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.743956 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.744463 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.744468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.744502 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.744963 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.744979 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.746809 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.746819 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.749679 (+ 2860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.749701 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.751257 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.751264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.751502 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.751549 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.751688 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.751873 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1075,"mutex_wait_us":84,"thread_start_us":1341,"threads_started":1}
I20260811 09:02:00.752490 18757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.742447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33527 (local address 127.9.40.66:44663)
0811 09:02:00.742664 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.742671 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.744311 (+ 1640us) server_negotiation.cc:407] Connection header received
0811 09:02:00.744524 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.744531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.744662 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.744800 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.745671 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.746870 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.749935 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.751247 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.751365 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.751468 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.751974 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.752161 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:02:00.753422 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.742758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44663 (local address 127.9.40.65:33527)
0811 09:02:00.744154 (+ 1396us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.744177 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.744324 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.744992 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.744995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.745021 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:00.745446 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.745459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.747072 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.747079 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.749686 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.749709 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.751548 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.751554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.751768 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.751802 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.752738 (+ 936us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.752909 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"mutex_wait_us":32,"thread_start_us":495,"threads_started":1}
I20260811 09:02:00.753607 18756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.742888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45965 (local address 127.9.40.67:46213)
0811 09:02:00.743089 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.743095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.743796 (+ 701us) server_negotiation.cc:407] Connection header received
0811 09:02:00.744186 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.744195 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.744306 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.744425 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:00.745171 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.746606 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.749932 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.750957 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.751170 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.751300 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.751742 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.751900 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:02:00.754379 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2859ce7a2246a0a7cbaea5140e8ee5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a" is_pre_election: true
I20260811 09:02:00.754702 18517 raft_consensus.cc:2454] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:00.754621 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2859ce7a2246a0a7cbaea5140e8ee5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1" is_pre_election: true
I20260811 09:02:00.755136 18678 raft_consensus.cc:2454] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:00.755389 18281 leader_election.cc:302] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e60ba19686914e048874af71180dd32d, fd2176f081ce4f588a56a3f745eab32a; no voters:
I20260811 09:02:00.755959 18768 raft_consensus.cc:2781] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:00.756132 18768 raft_consensus.cc:482] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:00.756249 18768 raft_consensus.cc:3037] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.758910 18768 raft_consensus.cc:504] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.759783 18768 leader_election.cc:288] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Requested vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:00.760493 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2859ce7a2246a0a7cbaea5140e8ee5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a"
I20260811 09:02:00.760607 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc2859ce7a2246a0a7cbaea5140e8ee5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1"
I20260811 09:02:00.760836 18517 raft_consensus.cc:3037] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.760900 18678 raft_consensus.cc:3037] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:00.764545 18517 raft_consensus.cc:2454] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:00.764896 18678 raft_consensus.cc:2454] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:00.765422 18281 leader_election.cc:302] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e60ba19686914e048874af71180dd32d, fd2176f081ce4f588a56a3f745eab32a; no voters:
I20260811 09:02:00.765821 18768 raft_consensus.cc:2781] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:00.766463 18768 raft_consensus.cc:686] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Becoming Leader. State: Replica: e60ba19686914e048874af71180dd32d, State: Running, Role: LEADER
I20260811 09:02:00.767007 18768 consensus_queue.cc:237] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.771520 18175 catalog_manager.cc:5495] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d reported cstate change: term changed from 0 to 1, leader changed from <none> to e60ba19686914e048874af71180dd32d (127.9.40.65). New cstate: current_term: 1 leader_uuid: "e60ba19686914e048874af71180dd32d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:02:00 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438917, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:02:00.862318 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.843470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38903 (local address 127.0.0.1:35158)
0811 09:02:00.844603 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:00.844622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:00.844843 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:00.845316 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:00.845320 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:00.846122 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:00.846383 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.846394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.848119 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.848124 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.849247 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:00.849262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.850343 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.850348 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:00.850523 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.851384 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:00.851411 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:00.854099 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:00.858694 (+ 4595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.858716 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.858724 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.859558 (+ 834us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.860055 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:00.860060 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:00.860064 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:00.860231 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:00.860943 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:00.860954 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:00.861179 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 09:02:00.861369 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 09:02:00.861763 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":978,"thread_start_us":739,"threads_started":1}
I20260811 09:02:00.862362 18801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:00.843592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.9.40.126:38903)
0811 09:02:00.844377 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:00.844385 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:00.844750 (+ 365us) server_negotiation.cc:407] Connection header received
0811 09:02:00.844997 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:00.845003 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:00.845094 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:00.845252 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:00.846613 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.847898 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.849433 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:00.850126 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:00.850294 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:00.854418 (+ 4124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:00.854427 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:00.854432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:00.854482 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:00.858441 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.859753 (+ 1312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.859757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.859759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.859913 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:00.860373 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:00.860383 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:00.860386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:00.860790 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:00.860978 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:00.861512 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 09:02:00.861923 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":169,"server-negotiator.queue_time_us":560,"thread_start_us":347,"threads_started":1}
I20260811 09:02:00.869657 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:35158:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:02:00.871191 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:00.881390 18495 tablet_service.cc:1467] Processing CreateTablet for tablet 61450ce83aff474987b16e0b52f54213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f81de3f972e4f698b632bbd10de1730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:02:00.881697 18337 tablet_service.cc:1467] Processing CreateTablet for tablet 61450ce83aff474987b16e0b52f54213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f81de3f972e4f698b632bbd10de1730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:02:00.882131 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61450ce83aff474987b16e0b52f54213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.882336 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61450ce83aff474987b16e0b52f54213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.885386 18655 tablet_service.cc:1467] Processing CreateTablet for tablet 61450ce83aff474987b16e0b52f54213 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=4f81de3f972e4f698b632bbd10de1730]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:02:00.886081 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61450ce83aff474987b16e0b52f54213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:00.891465 18760 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: Bootstrap starting.
I20260811 09:02:00.893608 18760 tablet_bootstrap.cc:654] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.897908 18761 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: Bootstrap starting.
I20260811 09:02:00.900202 18761 tablet_bootstrap.cc:654] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.903223 18763 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: Bootstrap starting.
I20260811 09:02:00.905293 18763 tablet_bootstrap.cc:654] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:00.914948 18763 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: No bootstrap required, opened a new log
I20260811 09:02:00.915283 18763 ts_tablet_manager.cc:1397] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 09:02:00.915866 18761 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: No bootstrap required, opened a new log
I20260811 09:02:00.916112 18761 ts_tablet_manager.cc:1397] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.011s
I20260811 09:02:00.917734 18761 raft_consensus.cc:348] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.916180 18763 raft_consensus.cc:348] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.918099 18761 raft_consensus.cc:374] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.918110 18763 raft_consensus.cc:374] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.918223 18763 raft_consensus.cc:729] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60ba19686914e048874af71180dd32d, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.918234 18761 raft_consensus.cc:729] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.918582 18761 consensus_queue.cc:260] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.918516 18763 consensus_queue.cc:260] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.919332 18761 ts_tablet_manager.cc:1428] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:02:00.919643 18768 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:00.919960 18768 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:00.920804 18760 tablet_bootstrap.cc:492] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: No bootstrap required, opened a new log
I20260811 09:02:00.921015 18760 ts_tablet_manager.cc:1397] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: Time spent bootstrapping tablet: real 0.030s user 0.010s sys 0.004s
I20260811 09:02:00.921963 18760 raft_consensus.cc:348] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.922277 18760 raft_consensus.cc:374] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:00.922363 18760 raft_consensus.cc:729] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2176f081ce4f588a56a3f745eab32a, State: Initialized, Role: FOLLOWER
I20260811 09:02:00.919133 18782 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:00.922726 18782 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:00.922991 18760 consensus_queue.cc:260] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:00.923787 18760 ts_tablet_manager.cc:1428] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:02:00.923571 18770 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:00.924041 18770 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:00.925751 18763 ts_tablet_manager.cc:1428] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260811 09:02:01.176573 18768 consensus_queue.cc:1035] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:01.177536 18768 consensus_queue.cc:1035] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:01.197683 18782 raft_consensus.cc:482] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:01.197996 18782 raft_consensus.cc:504] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.198734 18782 leader_election.cc:288] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e60ba19686914e048874af71180dd32d (127.9.40.65:45633), fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663)
I20260811 09:02:01.199486 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61450ce83aff474987b16e0b52f54213" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60ba19686914e048874af71180dd32d" is_pre_election: true
I20260811 09:02:01.199515 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61450ce83aff474987b16e0b52f54213" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a" is_pre_election: true
I20260811 09:02:01.199781 18359 raft_consensus.cc:2454] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 0.
I20260811 09:02:01.199803 18517 raft_consensus.cc:2454] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 0.
I20260811 09:02:01.200321 18600 leader_election.cc:302] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, e60ba19686914e048874af71180dd32d; no voters:
I20260811 09:02:01.200673 18782 raft_consensus.cc:2781] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:01.200845 18782 raft_consensus.cc:482] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:01.200953 18782 raft_consensus.cc:3037] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.203114 18782 raft_consensus.cc:504] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.203776 18782 leader_election.cc:288] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 election: Requested vote from peers e60ba19686914e048874af71180dd32d (127.9.40.65:45633), fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663)
I20260811 09:02:01.204538 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61450ce83aff474987b16e0b52f54213" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a"
I20260811 09:02:01.204572 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61450ce83aff474987b16e0b52f54213" candidate_uuid: "a2c278560eaf476da7697747d587fdb1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e60ba19686914e048874af71180dd32d"
I20260811 09:02:01.204836 18359 raft_consensus.cc:3037] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.204857 18517 raft_consensus.cc:3037] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.208063 18359 raft_consensus.cc:2454] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 1.
I20260811 09:02:01.208042 18517 raft_consensus.cc:2454] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2c278560eaf476da7697747d587fdb1 in term 1.
I20260811 09:02:01.208701 18600 leader_election.cc:302] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, e60ba19686914e048874af71180dd32d; no voters:
I20260811 09:02:01.209144 18782 raft_consensus.cc:2781] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:01.209784 18782 raft_consensus.cc:686] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [term 1 LEADER]: Becoming Leader. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Running, Role: LEADER
I20260811 09:02:01.210158 18782 consensus_queue.cc:237] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.212026 18811 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: TxnStatusTablet state changed. Reason: New leader a2c278560eaf476da7697747d587fdb1. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.212468 18811 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:02:01.213207 18813 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:02:01.213459 18813 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:02:01.213666 18175 catalog_manager.cc:5495] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2c278560eaf476da7697747d587fdb1 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } health_report { overall_health: HEALTHY } } }
I20260811 09:02:01.293442 18800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.286698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46213 (local address 127.0.0.1:50716)
0811 09:02:01.287046 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:01.287076 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:01.287261 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:01.287871 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:01.287875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:01.287887 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:02:01.288064 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.288072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.289781 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.289786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.290830 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.290844 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.291958 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.291963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.292171 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.292297 (+ 126us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.292877 (+ 580us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.292905 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:02:01.292989 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:02:01.293136 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 09:02:01.293499 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.286695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50716 (local address 127.9.40.67:46213)
0811 09:02:01.287461 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.287469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.287492 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:02:01.287572 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.287578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.287684 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.287812 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:01.288240 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.289606 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.290993 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.291734 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.291888 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.292565 (+ 677us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.292740 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.292838 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.293057 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.293158 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":432,"threads_started":1}
I20260811 09:02:01.298545 18359 raft_consensus.cc:1261] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Refusing update from remote peer a2c278560eaf476da7697747d587fdb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:01.298663 18517 raft_consensus.cc:1261] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Refusing update from remote peer a2c278560eaf476da7697747d587fdb1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:01.299335 18782 consensus_queue.cc:1035] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:01.299718 18811 consensus_queue.cc:1035] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:01.303838 18770 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: TxnStatusTablet state changed. Reason: New leader a2c278560eaf476da7697747d587fdb1. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.304191 18770 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:01.306077 18809 mvcc.cc:205] Tried to move back new op lower bound from 7317253821630545920 to 7317253821279305728. Current Snapshot: MvccSnapshot[applied={T|T < 7317253821630545920}]
I20260811 09:02:01.311537 18770 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.311933 18770 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P fd2176f081ce4f588a56a3f745eab32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:01.312676 18807 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: TxnStatusTablet state changed. Reason: New leader a2c278560eaf476da7697747d587fdb1. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.313050 18807 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:01.316340 18782 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.316774 18782 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:02:01.329582 18811 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.333928 18807 tablet_replica.cc:439] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a2c278560eaf476da7697747d587fdb1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } } }
I20260811 09:02:01.334275 18807 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P e60ba19686914e048874af71180dd32d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:02:01.334146 18811 tablet_replica.cc:442] T 61450ce83aff474987b16e0b52f54213 P a2c278560eaf476da7697747d587fdb1: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:02:01.341292 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.330857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58054 (local address 127.9.40.65:45633)
0811 09:02:01.334022 (+ 3165us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.334029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.334054 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:02:01.334173 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.334179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.334340 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.334689 (+ 349us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:01.335440 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.336655 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.338494 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.339038 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.339208 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.339779 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.340457 (+ 678us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.340644 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.340795 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.340909 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3001,"thread_start_us":2857,"threads_started":1}
I20260811 09:02:01.361606 18830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.354264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46213 (local address 127.9.40.65:36645)
0811 09:02:01.355161 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:01.355181 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:01.355343 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:01.355769 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:01.355773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:01.355783 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:02:01.356138 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.356146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.357791 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.357796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.359066 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.359086 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.360208 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.360215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.360365 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.360459 (+ 94us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.361021 (+ 562us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.361039 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 09:02:01.361136 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:02:01.361290 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":453,"threads_started":1}
I20260811 09:02:01.361627 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.354293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36645 (local address 127.9.40.67:46213)
0811 09:02:01.354513 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.354520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.355291 (+ 771us) server_negotiation.cc:407] Connection header received
0811 09:02:01.355508 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.355514 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.355622 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.355739 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:01.356311 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.357587 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.359257 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.359980 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.360139 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.360668 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.360874 (+ 206us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.360973 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.361181 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.361285 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:02:01.407996 18826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.400545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59671 (local address 127.9.40.65:45633)
0811 09:02:01.400824 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.400834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.401470 (+ 636us) server_negotiation.cc:407] Connection header received
0811 09:02:01.401719 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.401724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.401819 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.401882 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:01.402848 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.403831 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.405670 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.406197 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.406328 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.407069 (+ 741us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.407229 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.407327 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.407550 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.407669 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:02:01.408993 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.400500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45633 (local address 127.9.40.67:59671)
0811 09:02:01.401342 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:01.401371 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:01.401544 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:01.402121 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:01.402132 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:01.402145 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:02:01.402604 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.402619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.404021 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.404027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.405496 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:01.405512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.406493 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.406500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:01.406713 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.406808 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.407385 (+ 577us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.407398 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:02:01.408316 (+ 918us) client_negotiation.cc:240] Negotiation successful
0811 09:02:01.408503 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":413,"threads_started":1}
I20260811 09:02:01.461176 18175 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:01.461500 18175 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:01.465711 18175 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Sending DeleteTablet for 3 replicas of tablet fc2859ce7a2246a0a7cbaea5140e8ee5
I20260811 09:02:01.466897 18337 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:01 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49488
I20260811 09:02:01.466899 18495 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:01 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:01.467808 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet fc2859ce7a2246a0a7cbaea5140e8ee5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:01 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49014
I20260811 09:02:01.492003 18837 tablet_replica.cc:330] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: stopping tablet replica
I20260811 09:02:01.492452 18837 raft_consensus.cc:2229] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:01.493101 18837 raft_consensus.cc:2258] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:01.494562 18837 ts_tablet_manager.cc:1905] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:01.498381 18835 tablet_replica.cc:330] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: stopping tablet replica
I20260811 09:02:01.498811 18835 raft_consensus.cc:2229] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:01.499182 18835 raft_consensus.cc:2258] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:01.500597 18835 ts_tablet_manager.cc:1905] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:01.513466 18836 tablet_replica.cc:330] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: stopping tablet replica
I20260811 09:02:01.513935 18836 raft_consensus.cc:2229] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:02:01.514755 18836 raft_consensus.cc:2258] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:01.515600 18835 ts_tablet_manager.cc:1918] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:01.517323 18837 ts_tablet_manager.cc:1918] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:01.517643 18837 log.cc:1196] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/fc2859ce7a2246a0a7cbaea5140e8ee5
I20260811 09:02:01.518287 18837 ts_tablet_manager.cc:1939] T fc2859ce7a2246a0a7cbaea5140e8ee5 P a2c278560eaf476da7697747d587fdb1: Deleting consensus metadata
I20260811 09:02:01.519825 18159 catalog_manager.cc:4841] TS a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213): tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (table test-table [id=f9536c8c9cda4f5684636ea05fc79b76]) successfully deleted
I20260811 09:02:01.520315 18835 log.cc:1196] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/fc2859ce7a2246a0a7cbaea5140e8ee5
I20260811 09:02:01.521169 18835 ts_tablet_manager.cc:1939] T fc2859ce7a2246a0a7cbaea5140e8ee5 P fd2176f081ce4f588a56a3f745eab32a: Deleting consensus metadata
I20260811 09:02:01.522619 18158 catalog_manager.cc:4841] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (table test-table [id=f9536c8c9cda4f5684636ea05fc79b76]) successfully deleted
I20260811 09:02:01.524447 18836 ts_tablet_manager.cc:1905] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:01.533324 18836 ts_tablet_manager.cc:1918] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:01.544925 18836 log.cc:1196] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/fc2859ce7a2246a0a7cbaea5140e8ee5
I20260811 09:02:01.545749 18836 ts_tablet_manager.cc:1939] T fc2859ce7a2246a0a7cbaea5140e8ee5 P e60ba19686914e048874af71180dd32d: Deleting consensus metadata
I20260811 09:02:01.547550 18158 catalog_manager.cc:4841] TS e60ba19686914e048874af71180dd32d (127.9.40.65:45633): tablet fc2859ce7a2246a0a7cbaea5140e8ee5 (table test-table [id=f9536c8c9cda4f5684636ea05fc79b76]) successfully deleted
Aug 11 09:02:01 dist-test-slave-cbsf krb5kdc[18121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260811 09:02:01.580225 9377 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.105s user 0.000s sys 0.024s
Aug 11 09:02:01 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:01 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:02:01.620565 18845 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:01.588138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.9.40.126:38903)
0811 09:02:01.604212 (+ 16074us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.604220 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.604243 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:02:01.604317 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.604322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.604410 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.604565 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:01.606762 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.608074 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.610388 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.611207 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.611486 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.620211 (+ 8725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35172: BlockingRecv error: recv got EOF from 127.0.0.1:35172 (error 108)
Metrics: {"server-negotiator.queue_time_us":15897,"thread_start_us":15739,"threads_started":1}
Aug 11 09:02:01 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260811 09:02:01.645960 18845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.623867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35184 (local address 127.9.40.126:38903)
0811 09:02:01.624086 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.624119 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.625419 (+ 1300us) server_negotiation.cc:407] Connection header received
0811 09:02:01.625756 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.625763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.625857 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.625940 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:01.628118 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.629416 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.631457 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.632158 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.632329 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.638212 (+ 5883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:01.638223 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:01.638228 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:01.638291 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:01.642395 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:01.643503 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:01.643510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:01.643514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:01.643702 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:01.644196 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:01.644201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:01.644205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:01.644614 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:01.644929 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.645305 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.645582 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:02:01.650816 18175 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:02:01.651624 18175 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:01.660656 18337 tablet_service.cc:1467] Processing CreateTablet for tablet b67fa755d6b24452a6b56b5c04873c19 (DEFAULT_TABLE table=test-table [id=92c56e6bdf144906b8db7c0138e47070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:01.661123 18655 tablet_service.cc:1467] Processing CreateTablet for tablet b67fa755d6b24452a6b56b5c04873c19 (DEFAULT_TABLE table=test-table [id=92c56e6bdf144906b8db7c0138e47070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:01.661269 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67fa755d6b24452a6b56b5c04873c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:01.661617 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67fa755d6b24452a6b56b5c04873c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:01.670708 18495 tablet_service.cc:1467] Processing CreateTablet for tablet b67fa755d6b24452a6b56b5c04873c19 (DEFAULT_TABLE table=test-table [id=92c56e6bdf144906b8db7c0138e47070]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:01.671278 18495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b67fa755d6b24452a6b56b5c04873c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:01.683178 18852 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Bootstrap starting.
I20260811 09:02:01.687211 18852 tablet_bootstrap.cc:654] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:01.690337 18853 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Bootstrap starting.
I20260811 09:02:01.692391 18853 tablet_bootstrap.cc:654] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:01.693169 18851 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Bootstrap starting.
I20260811 09:02:01.695454 18851 tablet_bootstrap.cc:654] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:01.700201 18853 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: No bootstrap required, opened a new log
I20260811 09:02:01.700479 18853 ts_tablet_manager.cc:1397] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 09:02:01.701442 18853 raft_consensus.cc:348] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.701866 18853 raft_consensus.cc:374] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:01.702003 18853 raft_consensus.cc:729] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2176f081ce4f588a56a3f745eab32a, State: Initialized, Role: FOLLOWER
I20260811 09:02:01.702353 18853 consensus_queue.cc:260] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.703090 18853 ts_tablet_manager.cc:1428] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:02:01.706310 18851 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: No bootstrap required, opened a new log
I20260811 09:02:01.706319 18852 tablet_bootstrap.cc:492] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: No bootstrap required, opened a new log
I20260811 09:02:01.706519 18852 ts_tablet_manager.cc:1397] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.008s
I20260811 09:02:01.706609 18851 ts_tablet_manager.cc:1397] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.005s
I20260811 09:02:01.707640 18852 raft_consensus.cc:348] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.707628 18851 raft_consensus.cc:348] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.707904 18851 raft_consensus.cc:374] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:01.707996 18851 raft_consensus.cc:729] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Initialized, Role: FOLLOWER
I20260811 09:02:01.708050 18852 raft_consensus.cc:374] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:01.708331 18852 raft_consensus.cc:729] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60ba19686914e048874af71180dd32d, State: Initialized, Role: FOLLOWER
I20260811 09:02:01.708331 18851 consensus_queue.cc:260] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.708806 18852 consensus_queue.cc:260] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.709169 18851 ts_tablet_manager.cc:1428] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:02:01.710704 18852 ts_tablet_manager.cc:1428] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:02:01.904608 18807 raft_consensus.cc:482] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:01.904923 18807 raft_consensus.cc:504] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.905843 18807 leader_election.cc:288] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:01.906561 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67fa755d6b24452a6b56b5c04873c19" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a" is_pre_election: true
I20260811 09:02:01.906565 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67fa755d6b24452a6b56b5c04873c19" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1" is_pre_election: true
I20260811 09:02:01.906867 18678 raft_consensus.cc:2454] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:01.906888 18517 raft_consensus.cc:2454] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:01.907526 18282 leader_election.cc:302] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, e60ba19686914e048874af71180dd32d; no voters:
I20260811 09:02:01.907884 18807 raft_consensus.cc:2781] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:01.908048 18807 raft_consensus.cc:482] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:01.908164 18807 raft_consensus.cc:3037] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.911768 18807 raft_consensus.cc:504] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.912753 18807 leader_election.cc:288] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Requested vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:01.913440 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67fa755d6b24452a6b56b5c04873c19" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1"
I20260811 09:02:01.913738 18678 raft_consensus.cc:3037] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.917004 18517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b67fa755d6b24452a6b56b5c04873c19" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a"
I20260811 09:02:01.917119 18678 raft_consensus.cc:2454] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:01.917336 18517 raft_consensus.cc:3037] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:01.917726 18282 leader_election.cc:302] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, e60ba19686914e048874af71180dd32d; no voters:
I20260811 09:02:01.918179 18807 raft_consensus.cc:2781] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:01.918386 18807 raft_consensus.cc:686] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Becoming Leader. State: Replica: e60ba19686914e048874af71180dd32d, State: Running, Role: LEADER
I20260811 09:02:01.918797 18807 consensus_queue.cc:237] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:01.920840 18517 raft_consensus.cc:2454] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:01.923059 18174 catalog_manager.cc:5495] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d reported cstate change: term changed from 0 to 1, leader changed from <none> to e60ba19686914e048874af71180dd32d (127.9.40.65). New cstate: current_term: 1 leader_uuid: "e60ba19686914e048874af71180dd32d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } health_report { overall_health: UNKNOWN } } }
I20260811 09:02:01.986856 18859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:01.979429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.9.40.65:45633)
0811 09:02:01.980107 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:01.980113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:01.980136 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:02:01.980213 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:01.980219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:01.980304 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:01.980414 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:01.980962 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.982201 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.984398 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:01.985162 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:01.985300 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:01.985944 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:01.986116 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:01.986204 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:01.986376 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 09:02:01.986473 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":1217,"threads_started":1}
I20260811 09:02:01.991848 18678 raft_consensus.cc:1261] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Refusing update from remote peer e60ba19686914e048874af71180dd32d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:01.991955 18517 raft_consensus.cc:1261] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Refusing update from remote peer e60ba19686914e048874af71180dd32d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:01.993037 18793 consensus_queue.cc:1035] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:01.993784 18807 consensus_queue.cc:1035] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:02.015753 18174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:02.021340 18174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:02.025939 18174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Sending DeleteTablet for 3 replicas of tablet b67fa755d6b24452a6b56b5c04873c19
I20260811 09:02:02.027176 18337 tablet_service.cc:1514] Processing DeleteTablet for tablet b67fa755d6b24452a6b56b5c04873c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49488
I20260811 09:02:02.027453 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet b67fa755d6b24452a6b56b5c04873c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49014
I20260811 09:02:02.027558 18836 tablet_replica.cc:330] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: stopping tablet replica
I20260811 09:02:02.027791 18836 raft_consensus.cc:2229] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:02:02.027828 18494 tablet_service.cc:1514] Processing DeleteTablet for tablet b67fa755d6b24452a6b56b5c04873c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:02.027899 18495 tablet_service.cc:1514] Processing DeleteTablet for tablet b67fa755d6b24452a6b56b5c04873c19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:02.028105 18870 tablet_replica.cc:330] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: stopping tablet replica
I20260811 09:02:02.028205 18836 raft_consensus.cc:2258] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.028318 18870 raft_consensus.cc:2229] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:02.028791 18870 raft_consensus.cc:2258] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.028815 18871 tablet_replica.cc:330] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: stopping tablet replica
I20260811 09:02:02.028997 18871 raft_consensus.cc:2229] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:02.029004 18174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 09:02:02.029289 18871 raft_consensus.cc:2258] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.029553 18836 ts_tablet_manager.cc:1905] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 09:02:02.029856 18174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:02.033812 18871 ts_tablet_manager.cc:1905] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:02.036046 18870 ts_tablet_manager.cc:1905] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:02.036343 18836 ts_tablet_manager.cc:1918] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:02.036625 18836 log.cc:1196] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/b67fa755d6b24452a6b56b5c04873c19
I20260811 09:02:02.037237 18337 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 (DEFAULT_TABLE table=test-table [id=de5b364371704d468d3e19ed4cb9d9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:02.037596 18655 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 (DEFAULT_TABLE table=test-table [id=de5b364371704d468d3e19ed4cb9d9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:02.037726 18337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c34b001cc48d89282bf989d4f57f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:02.037878 18494 tablet_service.cc:1467] Processing CreateTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 (DEFAULT_TABLE table=test-table [id=de5b364371704d468d3e19ed4cb9d9b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:02.038020 18655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c34b001cc48d89282bf989d4f57f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:02.038322 18494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f5c34b001cc48d89282bf989d4f57f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:02.043017 18853 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Bootstrap starting.
I20260811 09:02:02.044410 18852 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Bootstrap starting.
I20260811 09:02:02.044483 18851 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Bootstrap starting.
I20260811 09:02:02.044739 18853 tablet_bootstrap.cc:654] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:02.046212 18852 tablet_bootstrap.cc:654] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:02.046435 18851 tablet_bootstrap.cc:654] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:02.051064 18836 ts_tablet_manager.cc:1939] T b67fa755d6b24452a6b56b5c04873c19 P e60ba19686914e048874af71180dd32d: Deleting consensus metadata
I20260811 09:02:02.052024 18871 ts_tablet_manager.cc:1918] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:02.052213 18871 log.cc:1196] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/b67fa755d6b24452a6b56b5c04873c19
I20260811 09:02:02.052599 18158 catalog_manager.cc:4841] TS e60ba19686914e048874af71180dd32d (127.9.40.65:45633): tablet b67fa755d6b24452a6b56b5c04873c19 (table test-table [id=92c56e6bdf144906b8db7c0138e47070]) successfully deleted
I20260811 09:02:02.052871 18871 ts_tablet_manager.cc:1939] T b67fa755d6b24452a6b56b5c04873c19 P fd2176f081ce4f588a56a3f745eab32a: Deleting consensus metadata
I20260811 09:02:02.054391 18158 catalog_manager.cc:4841] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): tablet b67fa755d6b24452a6b56b5c04873c19 (table test-table [id=92c56e6bdf144906b8db7c0138e47070]) successfully deleted
W20260811 09:02:02.054993 18158 catalog_manager.cc:4808] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): delete failed for tablet b67fa755d6b24452a6b56b5c04873c19 because the tablet was not found. No further retry: Not found: Tablet not found: b67fa755d6b24452a6b56b5c04873c19
I20260811 09:02:02.060765 18853 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: No bootstrap required, opened a new log
I20260811 09:02:02.060775 18851 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: No bootstrap required, opened a new log
I20260811 09:02:02.061005 18853 ts_tablet_manager.cc:1397] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.016s
I20260811 09:02:02.061246 18851 ts_tablet_manager.cc:1397] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.004s
I20260811 09:02:02.061868 18853 raft_consensus.cc:348] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.062177 18853 raft_consensus.cc:374] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:02.062278 18853 raft_consensus.cc:729] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2176f081ce4f588a56a3f745eab32a, State: Initialized, Role: FOLLOWER
I20260811 09:02:02.062325 18852 tablet_bootstrap.cc:492] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: No bootstrap required, opened a new log
I20260811 09:02:02.062320 18870 ts_tablet_manager.cc:1918] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:02.062460 18870 log.cc:1196] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/b67fa755d6b24452a6b56b5c04873c19
I20260811 09:02:02.062530 18852 ts_tablet_manager.cc:1397] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.003s
I20260811 09:02:02.062631 18853 consensus_queue.cc:260] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.062506 18851 raft_consensus.cc:348] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.063019 18870 ts_tablet_manager.cc:1939] T b67fa755d6b24452a6b56b5c04873c19 P a2c278560eaf476da7697747d587fdb1: Deleting consensus metadata
I20260811 09:02:02.063186 18851 raft_consensus.cc:374] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:02.063341 18851 raft_consensus.cc:729] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c278560eaf476da7697747d587fdb1, State: Initialized, Role: FOLLOWER
I20260811 09:02:02.063464 18852 raft_consensus.cc:348] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.063766 18852 raft_consensus.cc:374] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:02.063868 18852 raft_consensus.cc:729] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e60ba19686914e048874af71180dd32d, State: Initialized, Role: FOLLOWER
I20260811 09:02:02.063798 18851 consensus_queue.cc:260] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.064424 18159 catalog_manager.cc:4841] TS a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213): tablet b67fa755d6b24452a6b56b5c04873c19 (table test-table [id=92c56e6bdf144906b8db7c0138e47070]) successfully deleted
I20260811 09:02:02.064677 18851 ts_tablet_manager.cc:1428] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:02:02.065027 18853 ts_tablet_manager.cc:1428] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:02:02.064169 18852 consensus_queue.cc:260] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.066066 18852 ts_tablet_manager.cc:1428] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 09:02:02.395648 18793 raft_consensus.cc:482] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:02.395921 18793 raft_consensus.cc:504] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.396731 18793 leader_election.cc:288] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:02.397464 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c34b001cc48d89282bf989d4f57f5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a" is_pre_election: true
I20260811 09:02:02.397519 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c34b001cc48d89282bf989d4f57f5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1" is_pre_election: true
I20260811 09:02:02.397820 18515 raft_consensus.cc:2454] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:02.397861 18678 raft_consensus.cc:2454] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 0.
I20260811 09:02:02.398389 18281 leader_election.cc:302] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e60ba19686914e048874af71180dd32d, fd2176f081ce4f588a56a3f745eab32a; no voters:
I20260811 09:02:02.398758 18793 raft_consensus.cc:2781] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:02.398885 18793 raft_consensus.cc:482] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:02.399010 18793 raft_consensus.cc:3037] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:02.401338 18793 raft_consensus.cc:504] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.402105 18793 leader_election.cc:288] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Requested vote from peers fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663), a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213)
I20260811 09:02:02.402753 18515 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c34b001cc48d89282bf989d4f57f5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2176f081ce4f588a56a3f745eab32a"
I20260811 09:02:02.402743 18678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f5c34b001cc48d89282bf989d4f57f5" candidate_uuid: "e60ba19686914e048874af71180dd32d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c278560eaf476da7697747d587fdb1"
I20260811 09:02:02.403012 18515 raft_consensus.cc:3037] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:02.403040 18678 raft_consensus.cc:3037] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:02.405321 18678 raft_consensus.cc:2454] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:02.405464 18515 raft_consensus.cc:2454] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e60ba19686914e048874af71180dd32d in term 1.
I20260811 09:02:02.405951 18282 leader_election.cc:302] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c278560eaf476da7697747d587fdb1, e60ba19686914e048874af71180dd32d; no voters:
I20260811 09:02:02.406322 18793 raft_consensus.cc:2781] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:02.406548 18793 raft_consensus.cc:686] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Becoming Leader. State: Replica: e60ba19686914e048874af71180dd32d, State: Running, Role: LEADER
I20260811 09:02:02.406883 18793 consensus_queue.cc:237] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } }
I20260811 09:02:02.410336 18174 catalog_manager.cc:5495] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d reported cstate change: term changed from 0 to 1, leader changed from <none> to e60ba19686914e048874af71180dd32d (127.9.40.65). New cstate: current_term: 1 leader_uuid: "e60ba19686914e048874af71180dd32d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e60ba19686914e048874af71180dd32d" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 } health_report { overall_health: UNKNOWN } } }
I20260811 09:02:02.452307 18678 raft_consensus.cc:1261] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Refusing update from remote peer e60ba19686914e048874af71180dd32d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:02.452323 18515 raft_consensus.cc:1261] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Refusing update from remote peer e60ba19686914e048874af71180dd32d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:02.453039 18868 consensus_queue.cc:1035] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c278560eaf476da7697747d587fdb1" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:02.453527 18793 consensus_queue.cc:1035] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd2176f081ce4f588a56a3f745eab32a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:02.515412 18174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:02.515663 18174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:02.519217 18174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc4746867fe34c4081a1d7af30cea312: Sending DeleteTablet for 3 replicas of tablet 4f5c34b001cc48d89282bf989d4f57f5
I20260811 09:02:02.520319 18337 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49488
I20260811 09:02:02.520440 18655 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49014
I20260811 09:02:02.520445 18494 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f5c34b001cc48d89282bf989d4f57f5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:02 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34666
I20260811 09:02:02.520682 18836 tablet_replica.cc:330] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: stopping tablet replica
I20260811 09:02:02.520905 18870 tablet_replica.cc:330] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: stopping tablet replica
I20260811 09:02:02.520921 18836 raft_consensus.cc:2229] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:02:02.521119 18870 raft_consensus.cc:2229] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:02.521273 18836 raft_consensus.cc:2258] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.521322 18870 raft_consensus.cc:2258] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.521569 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 18262
I20260811 09:02:02.521632 18871 tablet_replica.cc:330] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: stopping tablet replica
I20260811 09:02:02.521845 18871 raft_consensus.cc:2229] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:02.522073 18871 raft_consensus.cc:2258] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:02.522748 18836 ts_tablet_manager.cc:1905] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:02.523327 18871 ts_tablet_manager.cc:1905] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:02.526562 18870 ts_tablet_manager.cc:1905] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:02.530617 18871 ts_tablet_manager.cc:1918] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:02.530712 18836 ts_tablet_manager.cc:1918] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:02.530854 18871 log.cc:1196] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/4f5c34b001cc48d89282bf989d4f57f5
I20260811 09:02:02.530855 18836 log.cc:1196] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/4f5c34b001cc48d89282bf989d4f57f5
I20260811 09:02:02.531349 18871 ts_tablet_manager.cc:1939] T 4f5c34b001cc48d89282bf989d4f57f5 P fd2176f081ce4f588a56a3f745eab32a: Deleting consensus metadata
I20260811 09:02:02.531371 18836 ts_tablet_manager.cc:1939] T 4f5c34b001cc48d89282bf989d4f57f5 P e60ba19686914e048874af71180dd32d: Deleting consensus metadata
I20260811 09:02:02.532819 18158 catalog_manager.cc:4841] TS fd2176f081ce4f588a56a3f745eab32a (127.9.40.66:44663): tablet 4f5c34b001cc48d89282bf989d4f57f5 (table test-table [id=de5b364371704d468d3e19ed4cb9d9b8]) successfully deleted
I20260811 09:02:02.534173 18158 catalog_manager.cc:4841] TS e60ba19686914e048874af71180dd32d (127.9.40.65:45633): tablet 4f5c34b001cc48d89282bf989d4f57f5 (table test-table [id=de5b364371704d468d3e19ed4cb9d9b8]) successfully deleted
I20260811 09:02:02.535262 18870 ts_tablet_manager.cc:1918] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:02:02.535511 18870 log.cc:1196] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/4f5c34b001cc48d89282bf989d4f57f5
Aug 11 09:02:02 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.65@KRBTEST.COM
I20260811 09:02:02.535964 18870 ts_tablet_manager.cc:1939] T 4f5c34b001cc48d89282bf989d4f57f5 P a2c278560eaf476da7697747d587fdb1: Deleting consensus metadata
I20260811 09:02:02.537249 18159 catalog_manager.cc:4841] TS a2c278560eaf476da7697747d587fdb1 (127.9.40.67:46213): tablet 4f5c34b001cc48d89282bf989d4f57f5 (table test-table [id=de5b364371704d468d3e19ed4cb9d9b8]) successfully deleted
I20260811 09:02:02.543737 18883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:02.524147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58066 (local address 127.9.40.65:45633)
0811 09:02:02.524661 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:02.524667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:02.524717 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:02:02.524816 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:02.524822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:02.524912 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:02.526961 (+ 2049us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:02.527243 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.528517 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.530634 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.531360 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.532224 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:02.537219 (+ 4995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:02.537242 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:02.537261 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:02.537325 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:02.540843 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.541721 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.541728 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.541731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.541884 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.542243 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.542247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.542249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.542682 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:02.542933 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:02.543228 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 09:02:02.543430 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":341,"thread_start_us":1337,"threads_started":1}
I20260811 09:02:02.544502 18410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:02.680124 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 18262
I20260811 09:02:02.712208 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 18420
Aug 11 09:02:02 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.66@KRBTEST.COM
I20260811 09:02:02.730924 18886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:02.713294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46964 (local address 127.9.40.66:44663)
0811 09:02:02.713817 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:02.713822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:02.713837 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:02:02.713896 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:02.713900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:02.713963 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:02.714122 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:02.715497 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.716352 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.717920 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.718371 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.718531 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:02.723549 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:02.723572 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:02.723591 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:02.723650 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:02.727222 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.728288 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.728305 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.728307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.728521 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.729010 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.729016 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.729019 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.729459 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:02.729812 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:02.730192 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 09:02:02.730551 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":273,"threads_started":1}
I20260811 09:02:02.731809 18568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:02.835067 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 18420
I20260811 09:02:02.860559 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 18578
Aug 11 09:02:02 dist-test-slave-cbsf krb5kdc[18121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438921, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.67@KRBTEST.COM
I20260811 09:02:02.879016 18888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:02.861505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50732 (local address 127.9.40.67:46213)
0811 09:02:02.862045 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:02.862050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:02.862069 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:02:02.862131 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:02.862134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:02.862199 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:02.862348 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:02.863797 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.864626 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.866236 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:02.866844 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:02.867014 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:02.872145 (+ 5131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:02.872165 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:02.872185 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:02.872238 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:02.875628 (+ 3390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.876663 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.876673 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.876675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.876956 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:02.877448 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:02.877452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:02.877454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:02.877897 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:02.878157 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:02.878475 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:02:02.878705 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":308,"threads_started":1}
I20260811 09:02:02.879863 18727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:03.039667 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 18578
I20260811 09:02:03.072062 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 18137
I20260811 09:02:03.073483 18221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:03.202163 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 18137
2026-08-11T09:02:03Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7248 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:02:03 dist-test-slave-cbsf krb5kdc[18899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:02:03 dist-test-slave-cbsf krb5kdc[18899](info): set up 2 sockets
Aug 11 09:02:03 dist-test-slave-cbsf krb5kdc[18899](info): commencing operation
krb5kdc: starting...
W20260811 09:02:04.442795 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:02:04 dist-test-slave-cbsf krb5kdc[18899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:02:04.754607 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40417
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:40117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:05.149685 18916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:05.150113 18916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:05.150301 18916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:05.158557 18916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:05.158792 18916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:05.158921 18916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:05.159031 18916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:05.159121 18916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:05.170288 18916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:05.171919 18916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:05.174245 18916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:05.188784 18923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:05.188839 18926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:05.188870 18924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:05.190160 18916 server_base.cc:1034] running on GCE node
I20260811 09:02:05.191181 18916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:05.193023 18916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:05.194317 18916 hybrid_clock.cc:648] HybridClock initialized: now 1786438925194270 us; error 43 us; skew 500 ppm
I20260811 09:02:05.194820 18916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T09:02:05Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2062 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:02:05 dist-test-slave-cbsf krb5kdc[18933](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:02:05 dist-test-slave-cbsf krb5kdc[18933](info): set up 2 sockets
Aug 11 09:02:05 dist-test-slave-cbsf krb5kdc[18933](info): commencing operation
krb5kdc: starting...
W20260811 09:02:06.509145 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.007s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:02:06 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:02:06.829370 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39607
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:07.216286 18948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:07.216751 18948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:07.216939 18948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:07.225024 18948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:07.225229 18948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:07.225354 18948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:07.225473 18948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:07.225560 18948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:07.236550 18948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:07.238255 18948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:07.239830 18948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:07.255060 18956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:07.255151 18955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:07.255060 18958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:07.257001 18948 server_base.cc:1034] running on GCE node
I20260811 09:02:07.258061 18948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:07.260073 18948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:07.261672 18948 hybrid_clock.cc:648] HybridClock initialized: now 1786438927261519 us; error 113 us; skew 500 ppm
I20260811 09:02:07.262203 18948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:07 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438927, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:07.266638 18948 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:02:07.269115 18948 webserver.cc:460] Webserver started at http://127.9.40.126:41509/ using document root <none> and password file <none>
I20260811 09:02:07.270081 18948 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:07.270282 18948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:07.270699 18948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:07.274694 18948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea"
format_stamp: "Formatted at 2026-08-11 09:02:07 on dist-test-slave-cbsf"
server_key: "ee72a3f0764d0feb45fb55e9c5725c2c"
server_key_iv: "e7181753cdf4a7da1b5c550934dbae76"
server_key_version: "encryptionkey@0"
I20260811 09:02:07.275774 18948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea"
format_stamp: "Formatted at 2026-08-11 09:02:07 on dist-test-slave-cbsf"
server_key: "ee72a3f0764d0feb45fb55e9c5725c2c"
server_key_iv: "e7181753cdf4a7da1b5c550934dbae76"
server_key_version: "encryptionkey@0"
I20260811 09:02:07.282998 18948 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 09:02:07.287978 18966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:07.290220 18948 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:07.290536 18948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea"
format_stamp: "Formatted at 2026-08-11 09:02:07 on dist-test-slave-cbsf"
server_key: "ee72a3f0764d0feb45fb55e9c5725c2c"
server_key_iv: "e7181753cdf4a7da1b5c550934dbae76"
server_key_version: "encryptionkey@0"
I20260811 09:02:07.290818 18948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:07.313830 18948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:07.319552 18948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:07.319947 18948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:07.341470 18948 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39607
I20260811 09:02:07.341491 19027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39607 every 8 connection(s)
I20260811 09:02:07.343220 18948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:02:07.348097 19028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:07.353852 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 18948
I20260811 09:02:07.354123 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:02:07.355105 9377 external_mini_cluster.cc:1428] Setting key c45889da5c6725c16fd17fc3ef587606
I20260811 09:02:07.363193 19028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Bootstrap starting.
I20260811 09:02:07.368084 19028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:07.369694 19028 log.cc:824] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:07.374246 19028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: No bootstrap required, opened a new log
Aug 11 09:02:07 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:07.383599 19028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:07.384130 19028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:07.384251 19028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec52ebf25074f9e8b76cb6fb4d3f0ea, State: Initialized, Role: FOLLOWER
I20260811 09:02:07.385010 19028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:07.385300 19028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:02:07.385448 19028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:02:07.385654 19028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:07.385821 19031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:07.357714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37714 (local address 127.9.40.126:39607)
0811 09:02:07.358701 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:07.358714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:07.358765 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:02:07.360983 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:07.361018 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:07.361538 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:07.361958 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:07.364496 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:07.366599 (+ 2103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:07.368985 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:07.369967 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:07.370158 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:07.375894 (+ 5736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:07.375905 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:07.375925 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:07.375979 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:07.380019 (+ 4040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:07.381195 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:07.381204 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:07.381213 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:07.381448 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:07.381964 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:07.381968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:07.381970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:07.382704 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:07.383083 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:07.383670 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 09:02:07.384288 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":460,"threads_started":1}
I20260811 09:02:07.388994 19028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:07.389742 19028 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec52ebf25074f9e8b76cb6fb4d3f0ea; no voters:
I20260811 09:02:07.390800 19028 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:02:07.390820 19033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:07.392051 19033 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 LEADER]: Becoming Leader. State: Replica: 2ec52ebf25074f9e8b76cb6fb4d3f0ea, State: Running, Role: LEADER
I20260811 09:02:07.392792 19033 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:07.394114 19028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:02:07.399326 19034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } } }
I20260811 09:02:07.399461 19035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec52ebf25074f9e8b76cb6fb4d3f0ea. Latest consensus state: current_term: 1 leader_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } } }
I20260811 09:02:07.399827 19034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:07.399968 19035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:07.404879 19042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:02:07.411543 19042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:02:07.423132 19042 catalog_manager.cc:1347] Generated new cluster ID: 7bb0c3cfddcb480ca3530c7abaa7a996
I20260811 09:02:07.423278 19042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:02:07.450963 19042 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:02:07.453133 19042 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:02:07.463709 19042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Generated new TSK 0
I20260811 09:02:07.464795 19042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:02:07.556301 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:07.956435 19058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:07.956892 19058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:07.957078 19058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:07.965297 19058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:07.965438 19058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:07.965632 19058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:02:07.976876 19058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:07.978470 19058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:07.980126 19058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:07.997149 19065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:07.996950 19068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:07.996994 19066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:07.998416 19058 server_base.cc:1034] running on GCE node
I20260811 09:02:07.999260 19058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:08.000586 19058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:08.001906 19058 hybrid_clock.cc:648] HybridClock initialized: now 1786438928001824 us; error 89 us; skew 500 ppm
I20260811 09:02:08.002363 19058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:08 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:08.006650 19058 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:02:08.008648 19058 webserver.cc:460] Webserver started at http://127.9.40.65:46715/ using document root <none> and password file <none>
I20260811 09:02:08.009618 19058 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:08.009820 19058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:08.010293 19058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:08.014743 19058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "1007542209504bdbadbcf521af32428e"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "f52e26b37e4c9aad5ddfc22660229ac2"
server_key_iv: "8f924355885450074b3f7c89db0aa01a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.015930 19058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "1007542209504bdbadbcf521af32428e"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "f52e26b37e4c9aad5ddfc22660229ac2"
server_key_iv: "8f924355885450074b3f7c89db0aa01a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.023106 19058 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.003s
I20260811 09:02:08.027701 19076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.029495 19058 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:08.029752 19058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "1007542209504bdbadbcf521af32428e"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "f52e26b37e4c9aad5ddfc22660229ac2"
server_key_iv: "8f924355885450074b3f7c89db0aa01a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.030011 19058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:08.073887 19058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:08.079180 19058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:08.079571 19058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:08.080822 19058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:08.082607 19058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:08.082744 19058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.082931 19058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:08.083017 19058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.123811 19058 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38977
I20260811 09:02:08.123971 19193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38977 every 8 connection(s)
I20260811 09:02:08.125403 19058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:02:08.133237 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19058
I20260811 09:02:08.133479 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:02:08.134209 9377 external_mini_cluster.cc:1428] Setting key df040c995466b08777f5e80c4a08b0e8
Aug 11 09:02:08 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:08.152999 19196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:08.128611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39529 (local address 127.9.40.126:39607)
0811 09:02:08.129224 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:08.129233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:08.130419 (+ 1186us) server_negotiation.cc:407] Connection header received
0811 09:02:08.132661 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:08.132670 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:08.132835 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:08.132997 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:08.135503 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.136557 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.138637 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.139476 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.139693 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:08.144082 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:08.144093 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:08.144102 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:08.144165 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:08.148760 (+ 4595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:08.150104 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:08.150111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:08.150115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:08.150309 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:08.150892 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:08.150898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:08.150901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:08.151240 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:08.151425 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:08.152272 (+ 847us) server_negotiation.cc:299] Negotiation successful
0811 09:02:08.152569 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":319,"threads_started":1}
I20260811 09:02:08.154237 19197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:08.129319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:39529)
0811 09:02:08.130197 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:08.130255 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:08.132193 (+ 1938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:08.133319 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:08.133370 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:08.134187 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:08.135294 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:08.135325 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.136776 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.136781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:08.138343 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:08.138364 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.139837 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.139844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:08.140113 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:08.141040 (+ 927us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:08.141069 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:08.143707 (+ 2638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:08.149043 (+ 5336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:08.149054 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:08.149073 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:08.149885 (+ 812us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:08.150465 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:08.150476 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:08.150478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:08.150750 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:08.151403 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:08.151412 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:08.151839 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 09:02:08.152161 (+ 322us) client_negotiation.cc:240] Negotiation successful
0811 09:02:08.152868 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":346,"threads_started":1}
I20260811 09:02:08.156242 19194 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39607
I20260811 09:02:08.156651 19194 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:08.157647 19194 heartbeater.cc:511] Master 127.9.40.126:39607 requested a full tablet report, sending...
I20260811 09:02:08.160423 18989 ts_manager.cc:194] Registered new tserver with Master: 1007542209504bdbadbcf521af32428e (127.9.40.65:38977)
I20260811 09:02:08.162146 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:39529
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:02:08.204257 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:08.590755 19200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:08.591168 19200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:08.591341 19200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:08.599365 19200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:08.599510 19200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:08.599637 19200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:02:08.610857 19200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:08.612257 19200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:08.613701 19200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:08.628574 19205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:08.628525 19206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:08.629391 19209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:08.630043 19200 server_base.cc:1034] running on GCE node
I20260811 09:02:08.631088 19200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:08.632571 19200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:08.633850 19200 hybrid_clock.cc:648] HybridClock initialized: now 1786438928633793 us; error 57 us; skew 500 ppm
I20260811 09:02:08.634258 19200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:08 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:08.641808 19200 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:02:08.643812 19200 webserver.cc:460] Webserver started at http://127.9.40.66:44357/ using document root <none> and password file <none>
I20260811 09:02:08.644742 19200 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:08.644941 19200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:08.645328 19200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:08.648916 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "a77cd7b537344199a0ccb3672fb38141"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "3b17481e4f4aecf3a009a458a45d7641"
server_key_iv: "64ef5d565220be4ae53afffed0e5127a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.649943 19200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "a77cd7b537344199a0ccb3672fb38141"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "3b17481e4f4aecf3a009a458a45d7641"
server_key_iv: "64ef5d565220be4ae53afffed0e5127a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.656466 19200 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:02:08.660986 19218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.662758 19200 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:02:08.663056 19200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "a77cd7b537344199a0ccb3672fb38141"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "3b17481e4f4aecf3a009a458a45d7641"
server_key_iv: "64ef5d565220be4ae53afffed0e5127a"
server_key_version: "encryptionkey@0"
I20260811 09:02:08.663317 19200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:08.681210 19200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:08.686280 19200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:08.686646 19200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:08.687832 19200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:08.689513 19200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:08.689633 19200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.689810 19200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:08.689888 19200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:08.728533 19200 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46433
I20260811 09:02:08.728552 19345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46433 every 8 connection(s)
I20260811 09:02:08.730110 19200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:02:08.739032 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19200
I20260811 09:02:08.739241 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:02:08.739894 9377 external_mini_cluster.cc:1428] Setting key 113d62346560c6d98a238e728e775c6b
Aug 11 09:02:08 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438928, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:08.754204 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:08.733184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42993 (local address 127.9.40.126:39607)
0811 09:02:08.733659 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:08.733664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:08.734992 (+ 1328us) server_negotiation.cc:407] Connection header received
0811 09:02:08.737256 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:08.737263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:08.737377 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:08.737508 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:08.739437 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.740544 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.742752 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.743230 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.743384 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:08.747569 (+ 4185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:08.747575 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:08.747579 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:08.747621 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:08.750559 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:08.751513 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:08.751519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:08.751521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:08.751655 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:08.752177 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:08.752182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:08.752184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:08.752499 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:08.752674 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:08.753515 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 09:02:08.753810 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":229,"threads_started":1}
I20260811 09:02:08.755311 19349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:08.733844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:42993)
0811 09:02:08.734735 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:08.734789 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:08.736871 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:08.737813 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:08.737824 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:08.738447 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:08.739155 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:08.739185 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.740799 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.740804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:08.742579 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:08.742595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:08.743514 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:08.743523 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:08.743795 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:08.744584 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:08.744611 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:08.747227 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:08.750775 (+ 3548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:08.750787 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:08.750819 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:08.751339 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:08.751782 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:08.751788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:08.751791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:08.752033 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:08.752645 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:08.752654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:08.753105 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 09:02:08.753404 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 09:02:08.754055 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":325,"threads_started":1}
I20260811 09:02:08.757282 19346 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39607
I20260811 09:02:08.757676 19346 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:08.758776 19346 heartbeater.cc:511] Master 127.9.40.126:39607 requested a full tablet report, sending...
I20260811 09:02:08.760921 18989 ts_manager.cc:194] Registered new tserver with Master: a77cd7b537344199a0ccb3672fb38141 (127.9.40.66:46433)
I20260811 09:02:08.761878 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:42993
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:02:08.807811 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:02:09.166311 19194 heartbeater.cc:503] Master 127.9.40.126:39607 was elected leader, sending a full tablet report...
W20260811 09:02:09.195716 19352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:09.196115 19352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:09.196238 19352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:09.204358 19352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:09.204501 19352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:09.204622 19352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:02:09.216193 19352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:09.217810 19352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:09.219463 19352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:09.236140 19360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:09.236079 19358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:09.236174 19357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:09.237229 19352 server_base.cc:1034] running on GCE node
I20260811 09:02:09.238023 19352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:09.239225 19352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:09.240604 19352 hybrid_clock.cc:648] HybridClock initialized: now 1786438929240530 us; error 65 us; skew 500 ppm
I20260811 09:02:09.241084 19352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:09 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:09.245117 19352 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:02:09.247133 19352 webserver.cc:460] Webserver started at http://127.9.40.67:38121/ using document root <none> and password file <none>
I20260811 09:02:09.248031 19352 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:09.248215 19352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:09.248601 19352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:09.252346 19352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "363defbac88842b49281304b73530332"
format_stamp: "Formatted at 2026-08-11 09:02:09 on dist-test-slave-cbsf"
server_key: "2a79e04fbbdc511b6900c57df3a96a8b"
server_key_iv: "9ff981656961a5680a42fd25f7d3745f"
server_key_version: "encryptionkey@0"
I20260811 09:02:09.253471 19352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "363defbac88842b49281304b73530332"
format_stamp: "Formatted at 2026-08-11 09:02:09 on dist-test-slave-cbsf"
server_key: "2a79e04fbbdc511b6900c57df3a96a8b"
server_key_iv: "9ff981656961a5680a42fd25f7d3745f"
server_key_version: "encryptionkey@0"
I20260811 09:02:09.260058 19352 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:02:09.264878 19370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:09.267058 19352 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:09.267313 19352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "363defbac88842b49281304b73530332"
format_stamp: "Formatted at 2026-08-11 09:02:09 on dist-test-slave-cbsf"
server_key: "2a79e04fbbdc511b6900c57df3a96a8b"
server_key_iv: "9ff981656961a5680a42fd25f7d3745f"
server_key_version: "encryptionkey@0"
I20260811 09:02:09.267565 19352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:09.285542 19352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:09.290812 19352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:09.291193 19352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:09.292377 19352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:09.294056 19352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:09.294145 19352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:09.294325 19352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:09.294390 19352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:09.333420 19352 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:39421
I20260811 09:02:09.333485 19487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:39421 every 8 connection(s)
I20260811 09:02:09.335028 19352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:02:09.342337 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19352
I20260811 09:02:09.342525 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:02:09.343106 9377 external_mini_cluster.cc:1428] Setting key 0053ca6591f67b31432aef57d98340a1
Aug 11 09:02:09 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438929, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:09.360299 19490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:09.338189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37741 (local address 127.9.40.126:39607)
0811 09:02:09.338795 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:09.338803 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:09.339927 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 09:02:09.342315 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:09.342321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:09.342463 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:09.342587 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:09.345190 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.346391 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.348488 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.349062 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.349220 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:09.353372 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:09.353378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:09.353382 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:09.353423 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:09.356175 (+ 2752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.357258 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.357262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.357264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.357405 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.358039 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.358043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.358045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.358406 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:09.358603 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:09.359591 (+ 988us) server_negotiation.cc:299] Negotiation successful
0811 09:02:09.359873 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":282,"threads_started":1}
I20260811 09:02:09.361539 19491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:09.338784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:37741)
0811 09:02:09.339700 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:09.339764 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:09.341925 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:09.343072 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:09.343083 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:09.343926 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:09.344965 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:09.344994 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.346659 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.346665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:09.348284 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:09.348303 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.349348 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.349358 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:09.349652 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:09.350377 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:09.350404 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:09.353049 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:09.356388 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:09.356398 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:09.356423 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:09.357083 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:09.357532 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:09.357537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:09.357540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:09.357880 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:09.358573 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:09.358584 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:09.359139 (+ 555us) client_negotiation.cc:769] Sending connection context
0811 09:02:09.359457 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 09:02:09.360100 (+ 643us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":353,"threads_started":1}
I20260811 09:02:09.363222 19488 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39607
I20260811 09:02:09.363600 19488 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:09.364440 19488 heartbeater.cc:511] Master 127.9.40.126:39607 requested a full tablet report, sending...
I20260811 09:02:09.366430 18989 ts_manager.cc:194] Registered new tserver with Master: 363defbac88842b49281304b73530332 (127.9.40.67:39421)
I20260811 09:02:09.367455 18989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:37741
I20260811 09:02:09.373004 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260811 09:02:09.559417 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19058
Aug 11 09:02:09 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:02:09.579550 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:09.560970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59942 (local address 127.9.40.65:38977)
0811 09:02:09.561738 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:09.561746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:09.561780 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:02:09.561869 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:09.561876 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:09.562109 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:09.562289 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:09.563933 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.565188 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.566830 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.567512 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.567651 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:09.572673 (+ 5022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:09.572740 (+ 67us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:09.572760 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:09.572813 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:09.575879 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.576944 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.576952 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.576955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.577110 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.577649 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.577653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.577655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.578092 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:09.578467 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:09.578828 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 09:02:09.579162 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":391,"threads_started":1}
I20260811 09:02:09.581063 19189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:09.674366 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19058
I20260811 09:02:09.702180 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19200
Aug 11 09:02:09 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:02:09.722168 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:09.703318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.9.40.66:46433)
0811 09:02:09.703930 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:09.703939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:09.703968 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:02:09.704036 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:09.704042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:09.704281 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:09.704526 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:09.706119 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.707246 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.708778 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.709433 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.709623 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:09.714931 (+ 5308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:09.714955 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:09.714975 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:09.715023 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:09.718605 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.719707 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.719717 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.719719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.719893 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.720307 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.720311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.720314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.720892 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:09.721177 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:09.721569 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 09:02:09.721822 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":276,"threads_started":1}
I20260811 09:02:09.723803 19340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:09.813048 19346 heartbeater.cc:503] Master 127.9.40.126:39607 was elected leader, sending a full tablet report...
I20260811 09:02:09.814740 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19200
W20260811 09:02:09.837458 18970 connection.cc:535] server connection from 127.9.40.66:42993 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:02:09.838037 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19352
Aug 11 09:02:09 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:02:09.857501 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:09.839203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.9.40.67:39421)
0811 09:02:09.839760 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:09.839768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:09.839801 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:02:09.839872 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:09.839879 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:09.840112 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:09.840314 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:09.841914 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.842978 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.844456 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:09.845216 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:09.845379 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:09.850415 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:09.850435 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:09.850455 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:09.850509 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:09.853952 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.854913 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.854923 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.854926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.855083 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:09.855515 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:09.855521 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:09.855524 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:09.855950 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:09.856295 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:09.856750 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 09:02:09.857070 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":256,"threads_started":1}
I20260811 09:02:09.859083 19483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:09.947917 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19352
I20260811 09:02:09.973052 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 18948
I20260811 09:02:09.974406 19023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:10.087793 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 18948
I20260811 09:02:10.117040 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39607
--webserver_interface=127.9.40.126
--webserver_port=41509
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:10.500078 19512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:10.500505 19512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:10.500721 19512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:10.508773 19512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:10.508965 19512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:10.509093 19512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:10.509202 19512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:10.509277 19512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:10.520318 19512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=41509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:10.521920 19512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:10.523423 19512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:10.536780 19519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:10.536975 19518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:10.537070 19521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:10.537990 19512 server_base.cc:1034] running on GCE node
I20260811 09:02:10.539049 19512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:10.540972 19512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:10.542245 19512 hybrid_clock.cc:648] HybridClock initialized: now 1786438930542164 us; error 71 us; skew 500 ppm
I20260811 09:02:10.542689 19512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:10 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438930, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:10.546599 19512 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:02:10.548552 19512 webserver.cc:460] Webserver started at http://127.9.40.126:41509/ using document root <none> and password file <none>
I20260811 09:02:10.549469 19512 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:10.549662 19512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:10.556269 19512 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 09:02:10.559980 19528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:10.561880 19512 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:10.562186 19512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea"
format_stamp: "Formatted at 2026-08-11 09:02:07 on dist-test-slave-cbsf"
server_key: "ee72a3f0764d0feb45fb55e9c5725c2c"
server_key_iv: "e7181753cdf4a7da1b5c550934dbae76"
server_key_version: "encryptionkey@0"
I20260811 09:02:10.563446 19512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:10.581666 19512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:10.587476 19512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:10.587864 19512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:10.611547 19512 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39607
I20260811 09:02:10.611553 19580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39607 every 8 connection(s)
I20260811 09:02:10.613636 19512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:02:10.620625 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19512
I20260811 09:02:10.621769 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:38977
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:39607
--webserver_port=46715
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:02:10.623250 19581 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:02:10.629292 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Bootstrap starting.
I20260811 09:02:10.654819 19581 log.cc:824] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:10.669431 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:02:10.670222 19581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Bootstrap complete.
I20260811 09:02:10.685173 19581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:10.686615 19581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ec52ebf25074f9e8b76cb6fb4d3f0ea, State: Initialized, Role: FOLLOWER
I20260811 09:02:10.687541 19581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:10.687940 19581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:02:10.688133 19581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:02:10.688431 19581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:02:10.695168 19581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:10.696113 19581 leader_election.cc:302] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ec52ebf25074f9e8b76cb6fb4d3f0ea; no voters:
I20260811 09:02:10.697175 19581 leader_election.cc:288] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:02:10.697315 19586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:02:10.698988 19586 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [term 2 LEADER]: Becoming Leader. State: Replica: 2ec52ebf25074f9e8b76cb6fb4d3f0ea, State: Running, Role: LEADER
I20260811 09:02:10.699749 19586 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } }
I20260811 09:02:10.700253 19581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:02:10.702966 19588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ec52ebf25074f9e8b76cb6fb4d3f0ea. Latest consensus state: current_term: 2 leader_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } } }
I20260811 09:02:10.702977 19587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ec52ebf25074f9e8b76cb6fb4d3f0ea" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39607 } } }
I20260811 09:02:10.703526 19587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:10.703527 19588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:10.716046 19593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:02:10.723999 19593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:02:10.726262 19593 catalog_manager.cc:1259] Loaded cluster ID: 7bb0c3cfddcb480ca3530c7abaa7a996
I20260811 09:02:10.726490 19593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:02:10.730738 19593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:02:10.734174 19593 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2ec52ebf25074f9e8b76cb6fb4d3f0ea: Loaded TSK: 0
I20260811 09:02:10.736989 19593 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:02:11.070855 19583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:11.071420 19583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:11.071645 19583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:11.080463 19583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:11.080682 19583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:11.080925 19583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:02:11.092891 19583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:38977
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=46715
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:11.094836 19583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:11.096401 19583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:11.111526 19611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:11.112387 19610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:11.112525 19613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:11.113001 19583 server_base.cc:1034] running on GCE node
I20260811 09:02:11.113802 19583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:11.115015 19583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:11.116258 19583 hybrid_clock.cc:648] HybridClock initialized: now 1786438931116214 us; error 43 us; skew 500 ppm
I20260811 09:02:11.116658 19583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438931, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:11.120849 19583 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260811 09:02:11.122655 19583 webserver.cc:460] Webserver started at http://127.9.40.65:46715/ using document root <none> and password file <none>
I20260811 09:02:11.123512 19583 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:11.123710 19583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:11.130291 19583 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:02:11.134486 19621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.136359 19583 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:02:11.136677 19583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "1007542209504bdbadbcf521af32428e"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "f52e26b37e4c9aad5ddfc22660229ac2"
server_key_iv: "8f924355885450074b3f7c89db0aa01a"
server_key_version: "encryptionkey@0"
I20260811 09:02:11.138028 19583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:11.167654 19583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:11.172983 19583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:11.173357 19583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:11.174580 19583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:11.176266 19583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:11.176414 19583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.176573 19583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:11.176666 19583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.215023 19583 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38977
I20260811 09:02:11.215095 19749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38977 every 8 connection(s)
I20260811 09:02:11.217113 19583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:02:11.220124 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19583
I20260811 09:02:11.220860 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:46433
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:39607
--webserver_port=44357
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:11.241168 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.221244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:53677)
0811 09:02:11.222391 (+ 1147us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.222452 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.225439 (+ 2987us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.227182 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.227196 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.228135 (+ 939us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.229079 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.229114 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.230887 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.230895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.233047 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.233074 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.234311 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.234320 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.234635 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.235416 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.235443 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.239629 (+ 4186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":475,"threads_started":1}
W20260811 09:02:11.241204 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.221443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53677 (local address 127.9.40.126:39607)
0811 09:02:11.225303 (+ 3860us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.225317 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.225367 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:02:11.226105 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.226118 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.226528 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.226870 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.229357 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.230660 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.233262 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.234011 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.234272 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.240073 (+ 5801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:53677: BlockingRecv error: recv got EOF from 127.9.40.65:53677 (error 108)
Metrics: {"server-negotiator.queue_time_us":3590,"thread_start_us":3371,"threads_started":1}
W20260811 09:02:11.242234 19751 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39607 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:39607: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:11.254107 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.243002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:41135)
0811 09:02:11.243333 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.243356 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.243497 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.243878 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.243887 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.244497 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.244798 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.244815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.245992 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.246002 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.247854 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.247879 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.248767 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.248778 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.249035 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.249765 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.249797 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.253762 (+ 3965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174}
W20260811 09:02:11.254458 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.243161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:41135 (local address 127.9.40.126:39607)
0811 09:02:11.243339 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.243343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.243412 (+ 69us) server_negotiation.cc:407] Connection header received
0811 09:02:11.243646 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.243650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.243714 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.243829 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.244974 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.245813 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.248068 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.248513 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.248645 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.254040 (+ 5395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:41135: BlockingRecv error: recv got EOF from 127.9.40.65:41135 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:11.264302 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.254953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:55521)
0811 09:02:11.255143 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.255165 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.255282 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.255633 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.255636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.256062 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.256243 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.256253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.257888 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.257892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.258879 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.258893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.259995 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.260000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.260161 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.260928 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.260948 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.264056 (+ 3108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 09:02:11.264740 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.255070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:55521 (local address 127.9.40.126:39607)
0811 09:02:11.255267 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.255274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.255297 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:02:11.255373 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.255379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.255459 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.255588 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.256415 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.257688 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.259044 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.259787 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.259930 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.264397 (+ 4467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:55521: BlockingRecv error: recv got EOF from 127.9.40.65:55521 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 09:02:11.264875 19751 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:02:11.600553 19757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:11.601014 19757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:11.601215 19757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:11.609074 19757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:11.609216 19757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:11.609338 19757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:02:11.625028 19757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:46433
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=44357
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:11.626863 19757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:11.628324 19757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:11.645231 19762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:11.645371 19763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:11.645416 19765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:11.646811 19757 server_base.cc:1034] running on GCE node
I20260811 09:02:11.647560 19757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:11.648658 19757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:11.649977 19757 hybrid_clock.cc:648] HybridClock initialized: now 1786438931649936 us; error 39 us; skew 500 ppm
I20260811 09:02:11.650386 19757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438931, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:11.654290 19757 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260811 09:02:11.656220 19757 webserver.cc:460] Webserver started at http://127.9.40.66:44357/ using document root <none> and password file <none>
I20260811 09:02:11.657212 19757 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:11.657402 19757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:11.664126 19757 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:02:11.667814 19774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.669898 19757 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 09:02:11.670166 19757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "a77cd7b537344199a0ccb3672fb38141"
format_stamp: "Formatted at 2026-08-11 09:02:08 on dist-test-slave-cbsf"
server_key: "3b17481e4f4aecf3a009a458a45d7641"
server_key_iv: "64ef5d565220be4ae53afffed0e5127a"
server_key_version: "encryptionkey@0"
I20260811 09:02:11.671397 19757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:11.686712 19757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:11.691571 19757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:11.691923 19757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:11.693189 19757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:11.694802 19757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:11.694962 19757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.695122 19757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:11.695225 19757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:11.735841 19757 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46433
I20260811 09:02:11.735911 19913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46433 every 8 connection(s)
I20260811 09:02:11.738155 19757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:02:11.747574 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19757
I20260811 09:02:11.748276 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:39421
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:39607
--webserver_port=38121
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:02:11.758350 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.741276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57765 (local address 127.9.40.126:39607)
0811 09:02:11.741534 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.741542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.743030 (+ 1488us) server_negotiation.cc:407] Connection header received
0811 09:02:11.745537 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.745544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.745670 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.745802 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.747777 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.748828 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.750865 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.751387 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.751562 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.758045 (+ 6483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:57765: BlockingRecv error: recv got EOF from 127.9.40.66:57765 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:02:11.759222 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.741811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:57765)
0811 09:02:11.742815 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.742868 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.745141 (+ 2273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.746123 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.746134 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.746798 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.747527 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.747548 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.749046 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.749052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.750638 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.750657 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.751711 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.751720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.752013 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.752901 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.752928 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.757782 (+ 4854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":396,"threads_started":1}
W20260811 09:02:11.760088 19914 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39607 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:39607: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:11.770861 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.760899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:38741)
0811 09:02:11.761150 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.761170 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.761280 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.761645 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.761649 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.762015 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.762192 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.762201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.763826 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.763830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.764969 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.764985 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.766169 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.766174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.766323 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.767061 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.767081 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.770630 (+ 3549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:02:11.771351 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.761014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:38741 (local address 127.9.40.126:39607)
0811 09:02:11.761255 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.761264 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.761290 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:02:11.761386 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.761393 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.761488 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.761606 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.762366 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.763658 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.765157 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.765944 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.766121 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.771001 (+ 4880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:38741: BlockingRecv error: recv got EOF from 127.9.40.66:38741 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:11 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:11.782936 19917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:11.771784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:37113)
0811 09:02:11.772052 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:11.772076 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:11.772208 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:11.772585 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:11.772593 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:11.773129 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:11.773405 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.773421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.774533 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.774541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.776416 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:11.776442 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.777461 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.777469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:11.777720 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.778425 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:11.778458 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:11.782592 (+ 4134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 09:02:11.783113 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:11.771958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:37113 (local address 127.9.40.126:39607)
0811 09:02:11.772154 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:11.772158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:11.772178 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:02:11.772343 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:11.772348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:11.772427 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:11.772537 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:11.773604 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.774365 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.776645 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:11.777283 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:11.777408 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:11.782859 (+ 5451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:37113: BlockingRecv error: recv got EOF from 127.9.40.66:37113 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:02:11.783553 19914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:02:12.131862 19918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:12.132197 19918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:12.132375 19918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:12.140406 19918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:12.140565 19918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:12.140791 19918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:02:12.151588 19918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:39421
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=38121
--tserver_master_addrs=127.9.40.126:39607
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:12.153308 19918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:12.154798 19918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:12.171131 19924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:12.171169 19932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:12.171216 19930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:12.172610 19918 server_base.cc:1034] running on GCE node
I20260811 09:02:12.173548 19918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:12.174723 19918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:12.175956 19918 hybrid_clock.cc:648] HybridClock initialized: now 1786438932175875 us; error 88 us; skew 500 ppm
I20260811 09:02:12.176402 19918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438932, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:12.180186 19918 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260811 09:02:12.182055 19918 webserver.cc:460] Webserver started at http://127.9.40.67:38121/ using document root <none> and password file <none>
I20260811 09:02:12.182885 19918 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:12.183090 19918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:12.189497 19918 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:02:12.192991 19940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:12.194743 19918 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:02:12.195017 19918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "363defbac88842b49281304b73530332"
format_stamp: "Formatted at 2026-08-11 09:02:09 on dist-test-slave-cbsf"
server_key: "2a79e04fbbdc511b6900c57df3a96a8b"
server_key_iv: "9ff981656961a5680a42fd25f7d3745f"
server_key_version: "encryptionkey@0"
I20260811 09:02:12.196169 19918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:12.216504 19918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:12.221987 19918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:12.222340 19918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:12.223479 19918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:12.225068 19918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:12.225203 19918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:12.225342 19918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:12.225446 19918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:12.264325 19918 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:39421
I20260811 09:02:12.264369 20064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:39421 every 8 connection(s)
I20260811 09:02:12.266575 19918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:12.275002 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 19918
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:12.277312 20065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.265617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:45361)
0811 09:02:12.266206 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:12.266230 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:12.266432 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:12.266954 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:12.266959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:12.267532 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:12.267931 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.267949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.269312 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.269318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.270659 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.270675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.271840 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.271848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.272017 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.272814 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:12.272834 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:12.276806 (+ 3972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":284,"threads_started":1}
W20260811 09:02:12.277761 19755 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:12.265746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45361 (local address 127.9.40.126:39607)
0811 09:02:12.266019 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.266026 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.266351 (+ 325us) server_negotiation.cc:407] Connection header received
0811 09:02:12.266634 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.266639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.266773 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.266912 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.268134 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.269152 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.270848 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.271596 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.271744 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.277098 (+ 5354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:45361: BlockingRecv error: recv got EOF from 127.9.40.65:45361 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:02:12.289678 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:12.269918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34655 (local address 127.9.40.126:39607)
0811 09:02:12.270567 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.270573 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.272307 (+ 1734us) server_negotiation.cc:407] Connection header received
0811 09:02:12.274218 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.274222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.274310 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.274450 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.277437 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.278617 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.281820 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.282372 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.282543 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.289336 (+ 6793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:34655: BlockingRecv error: recv got EOF from 127.9.40.67:34655 (error 108)
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":314,"threads_started":1}
I20260811 09:02:12.290488 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.270501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:34655)
0811 09:02:12.271495 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:12.271573 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:12.273794 (+ 2221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:12.274803 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:12.274819 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:12.275763 (+ 944us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:12.276864 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.276898 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.278887 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.278897 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.281604 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.281626 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.282678 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.282689 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.283009 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.283829 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:12.283855 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:12.289043 (+ 5188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":340,"threads_started":1}
W20260811 09:02:12.291298 20066 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39607 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:39607: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
I20260811 09:02:12.299952 20070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.276155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37354 (local address 127.9.40.126:39607)
0811 09:02:12.277558 (+ 1403us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.277565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.281460 (+ 3895us) server_negotiation.cc:407] Connection header received
0811 09:02:12.281730 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.281736 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.281824 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.281960 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.283917 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.285033 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.287042 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.287575 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.287734 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.290896 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:12.290905 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:12.290924 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:12.290982 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:12.295390 (+ 4408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:12.296477 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:12.296485 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:12.296494 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:12.296856 (+ 362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:12.297681 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:12.297684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:12.297687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:12.298239 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:12.298595 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:12.299182 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 09:02:12.299639 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1228,"thread_start_us":1047,"threads_started":1}
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:12.302675 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.291982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:56725)
0811 09:02:12.292240 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:12.292262 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:12.292390 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:12.292836 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:12.292840 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:12.293326 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:12.293505 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.293517 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.295110 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.295114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.296487 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.296503 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.297591 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.297603 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.297935 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.298630 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:12.298649 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:12.302353 (+ 3704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 09:02:12.302888 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:12.292144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56725 (local address 127.9.40.126:39607)
0811 09:02:12.292468 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.292474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.292494 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:02:12.292561 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.292577 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.292660 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.292800 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.293685 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.294904 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.296665 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.297285 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.297384 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.302620 (+ 5236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:56725: BlockingRecv error: recv got EOF from 127.9.40.67:56725 (error 108)
Metrics: {"server-negotiator.queue_time_us":182}
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:12.312939 20069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.303689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:54451)
0811 09:02:12.303881 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:12.303904 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:12.304029 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:12.304535 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:12.304539 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:12.305059 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:12.305322 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.305336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.306503 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.306507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.307482 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.307494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.308291 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.308296 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.308457 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.309221 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:12.309241 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:12.312494 (+ 3253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 09:02:12.313316 20066 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:02:12.313516 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:12.303973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54451 (local address 127.9.40.126:39607)
0811 09:02:12.304186 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.304191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.304207 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:02:12.304257 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.304260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.304384 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.304490 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.305530 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.306350 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.307643 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.308129 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.308247 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.312897 (+ 4650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:54451: BlockingRecv error: recv got EOF from 127.9.40.67:54451 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:12 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:12.793800 20072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:12.784225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:36947)
0811 09:02:12.784852 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:12.784869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:12.785050 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:12.785422 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:12.785427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:12.785983 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:12.786190 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.786200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.787265 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.787271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.788305 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:12.788319 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.789229 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.789236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:12.789429 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.790066 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:12.790085 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:12.793441 (+ 3356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":308,"threads_started":1}
W20260811 09:02:12.794034 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:12.784307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:36947 (local address 127.9.40.126:39607)
0811 09:02:12.784607 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:12.784613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:12.784966 (+ 353us) server_negotiation.cc:407] Connection header received
0811 09:02:12.785199 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:12.785203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:12.785274 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:12.785387 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:12.786359 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.787099 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.788509 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:12.789015 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:12.789165 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:12.793788 (+ 4623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:36947: BlockingRecv error: recv got EOF from 127.9.40.66:36947 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:13.288952 20073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:13.278513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:44805)
0811 09:02:13.279092 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:13.279112 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:13.279261 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:13.279610 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:13.279614 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:13.280096 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:13.280264 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.280274 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.281543 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.281548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.282652 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.282669 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.283599 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.283604 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.283741 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.284603 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:13.284625 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:13.288539 (+ 3914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":315,"threads_started":1}
W20260811 09:02:13.289109 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:13.278599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44805 (local address 127.9.40.126:39607)
0811 09:02:13.278863 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:13.278871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:13.279208 (+ 337us) server_negotiation.cc:407] Connection header received
0811 09:02:13.279375 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:13.279379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:13.279450 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:13.279566 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:13.280438 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.281309 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.282850 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.283399 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.283521 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.288879 (+ 5358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:44805: BlockingRecv error: recv got EOF from 127.9.40.65:44805 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:13.323939 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:13.314273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:53481)
0811 09:02:13.314953 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:13.314978 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:13.315133 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:13.315488 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:13.315492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:13.316004 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:13.316211 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.316222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.317578 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.317582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.318628 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.318644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.319516 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.319521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.319670 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.320389 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:13.320412 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:13.323684 (+ 3272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":359,"threads_started":1}
W20260811 09:02:13.324240 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:13.314340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53481 (local address 127.9.40.126:39607)
0811 09:02:13.314560 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:13.314583 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:13.315087 (+ 504us) server_negotiation.cc:407] Connection header received
0811 09:02:13.315246 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:13.315249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:13.315319 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:13.315407 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:13.316468 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.317414 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.318793 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.319313 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.319426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.323974 (+ 4548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:53481: BlockingRecv error: recv got EOF from 127.9.40.67:53481 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:13 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:13.808786 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:13.794977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:47275)
0811 09:02:13.795586 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:13.795603 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:13.795783 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:13.796168 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:13.796173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:13.796644 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:13.796829 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.796838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.797951 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.797959 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.798905 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:13.798918 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.799740 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.799746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:13.799936 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.805291 (+ 5355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:13.805315 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:13.808453 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":345,"threads_started":1}
W20260811 09:02:13.809098 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:13.795104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:47275 (local address 127.9.40.126:39607)
0811 09:02:13.795370 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:13.795378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:13.795707 (+ 329us) server_negotiation.cc:407] Connection header received
0811 09:02:13.795921 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:13.795927 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:13.796009 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:13.796127 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:13.796990 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.797795 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.799082 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:13.799562 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:13.799709 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:13.808766 (+ 9057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:47275: BlockingRecv error: recv got EOF from 127.9.40.66:47275 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:14.300802 20077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:14.290325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:60501)
0811 09:02:14.291025 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:14.291052 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:14.291193 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:14.291541 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:14.291545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:14.291915 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:14.292108 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.292116 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.293383 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.293389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.294685 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.294708 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.295698 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.295705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.295905 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.296837 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:14.296858 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:14.300393 (+ 3535us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":400,"threads_started":1}
W20260811 09:02:14.301216 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:14.290613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60501 (local address 127.9.40.126:39607)
0811 09:02:14.290870 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:14.290875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:14.291187 (+ 312us) server_negotiation.cc:407] Connection header received
0811 09:02:14.291311 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:14.291315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:14.291388 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:14.291493 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:14.292273 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.293148 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.294889 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.295504 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.295613 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.300635 (+ 5022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:60501: BlockingRecv error: recv got EOF from 127.9.40.65:60501 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:14.334641 20078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:14.325216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:49871)
0811 09:02:14.325868 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:14.325885 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:14.326030 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:14.326427 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:14.326431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:14.326914 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:14.327207 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.327224 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.328356 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.328360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.329348 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.329363 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.330089 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.330093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.330215 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.330869 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:14.330889 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:14.334267 (+ 3378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":379,"threads_started":1}
W20260811 09:02:14.334838 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:14.325303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49871 (local address 127.9.40.126:39607)
0811 09:02:14.325503 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:14.325507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:14.325994 (+ 487us) server_negotiation.cc:407] Connection header received
0811 09:02:14.326161 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:14.326166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:14.326248 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:14.326342 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:14.327407 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.328190 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.329501 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.329922 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.330038 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.334539 (+ 4501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:49871: BlockingRecv error: recv got EOF from 127.9.40.67:49871 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:14 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:14.819777 20080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:14.810233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:59679)
0811 09:02:14.810833 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:14.810852 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:14.810980 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:14.811365 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:14.811372 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:14.811859 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:14.812071 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.812082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.813324 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.813330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.814302 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:14.814316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.815097 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.815104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:14.815297 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.815976 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:14.815995 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:14.819364 (+ 3369us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":326,"threads_started":1}
W20260811 09:02:14.819964 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:14.810327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59679 (local address 127.9.40.126:39607)
0811 09:02:14.810552 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:14.810559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:14.810948 (+ 389us) server_negotiation.cc:407] Connection header received
0811 09:02:14.811093 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:14.811100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:14.811190 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:14.811313 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:14.812265 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.813163 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.814479 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:14.814928 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:14.815073 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:14.819660 (+ 4587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:59679: BlockingRecv error: recv got EOF from 127.9.40.66:59679 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:15.311122 20081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:15.301837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:44631)
0811 09:02:15.302418 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:15.302435 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:15.302574 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:15.302955 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:15.302959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:15.303408 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:15.303588 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.303598 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.304640 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.304644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.305745 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.305760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.306554 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.306559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.306704 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.307358 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:15.307381 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:15.310771 (+ 3390us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":360,"threads_started":1}
W20260811 09:02:15.311362 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:15.302003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44631 (local address 127.9.40.126:39607)
0811 09:02:15.302261 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:15.302268 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:15.302526 (+ 258us) server_negotiation.cc:407] Connection header received
0811 09:02:15.302687 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:15.302691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:15.302760 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:15.302862 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:15.303736 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.304479 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.305910 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.306372 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.306482 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.311032 (+ 4550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:44631: BlockingRecv error: recv got EOF from 127.9.40.65:44631 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:15.345914 20082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:15.336113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:58143)
0811 09:02:15.336875 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:15.336893 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:15.337015 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:15.337542 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:15.337547 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:15.338065 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:15.338339 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.338352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.339453 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.339461 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.340397 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.340409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.341250 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.341256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.341443 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.342146 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:15.342175 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:15.345632 (+ 3457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":447,"threads_started":1}
W20260811 09:02:15.346220 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:15.336141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58143 (local address 127.9.40.126:39607)
0811 09:02:15.336347 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:15.336352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:15.336964 (+ 612us) server_negotiation.cc:407] Connection header received
0811 09:02:15.337292 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:15.337300 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:15.337385 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:15.337507 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:15.338501 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.339308 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.340555 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.341086 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.341232 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.345929 (+ 4697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:58143: BlockingRecv error: recv got EOF from 127.9.40.67:58143 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:15 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:15.831799 20084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:15.821016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:43959)
0811 09:02:15.821652 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:15.821681 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:15.821841 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:15.822219 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:15.822223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:15.822685 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:15.822873 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.822884 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.824186 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.824192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.825720 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:15.825738 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.826662 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.826669 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:15.826893 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.827694 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:15.827724 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:15.831380 (+ 3656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":372,"threads_started":1}
W20260811 09:02:15.832015 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:15.821203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43959 (local address 127.9.40.126:39607)
0811 09:02:15.821414 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:15.821420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:15.821842 (+ 422us) server_negotiation.cc:407] Connection header received
0811 09:02:15.821945 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:15.821949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:15.822024 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:15.822124 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:15.823060 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.823955 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.825906 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:15.826480 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:15.826640 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:15.831712 (+ 5072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:43959: BlockingRecv error: recv got EOF from 127.9.40.66:43959 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:16.321879 20085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:16.312282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:53725)
0811 09:02:16.312933 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:16.312953 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:16.313084 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:16.313399 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:16.313402 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:16.313848 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:16.314027 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.314037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.315155 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.315160 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.316214 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.316229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.317236 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.317245 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.317412 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.318201 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:16.318222 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:16.321574 (+ 3352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":304,"threads_started":1}
W20260811 09:02:16.322167 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:16.312497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53725 (local address 127.9.40.126:39607)
0811 09:02:16.312771 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:16.312778 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:16.313039 (+ 261us) server_negotiation.cc:407] Connection header received
0811 09:02:16.313189 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:16.313193 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:16.313258 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:16.313354 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:16.314188 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.314980 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.316383 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.316969 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.317172 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.321901 (+ 4729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:53725: BlockingRecv error: recv got EOF from 127.9.40.65:53725 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:16.357270 20086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:16.347143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:59131)
0811 09:02:16.347821 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:16.347841 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:16.347974 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:16.348422 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:16.348427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:16.349030 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:16.349286 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.349300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.350548 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.350577 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.351734 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.351750 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.352581 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.352586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.352778 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.353445 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:16.353466 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:16.356958 (+ 3492us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":309,"threads_started":1}
W20260811 09:02:16.357592 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:16.347393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59131 (local address 127.9.40.126:39607)
0811 09:02:16.347623 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:16.347629 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:16.347933 (+ 304us) server_negotiation.cc:407] Connection header received
0811 09:02:16.348127 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:16.348132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:16.348200 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:16.348338 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:16.349451 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.350381 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.351918 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.352396 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.352507 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.357248 (+ 4741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:59131: BlockingRecv error: recv got EOF from 127.9.40.67:59131 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:16 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:16.842440 20087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:16.833156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:41619)
0811 09:02:16.833755 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:16.833776 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:16.833923 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:16.834319 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:16.834325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:16.834791 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:16.834959 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.834968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.836048 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.836055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.837051 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:16.837065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.837858 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.837864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:16.838059 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.838771 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:16.838791 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:16.841974 (+ 3183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":471,"threads_started":1}
W20260811 09:02:16.842803 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:16.833416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:41619 (local address 127.9.40.126:39607)
0811 09:02:16.833633 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:16.833640 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:16.833895 (+ 255us) server_negotiation.cc:407] Connection header received
0811 09:02:16.834076 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:16.834082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:16.834158 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:16.834278 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:16.835128 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.835877 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.837250 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:16.837692 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:16.837838 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:16.842278 (+ 4440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:41619: BlockingRecv error: recv got EOF from 127.9.40.66:41619 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:17.332362 20088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:17.323018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:60253)
0811 09:02:17.323598 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:17.323614 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:17.323789 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:17.324155 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:17.324160 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:17.324632 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:17.324844 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.324854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.325991 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.325997 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.326950 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.326963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.327790 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.327797 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.327988 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.328676 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:17.328720 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:17.331989 (+ 3269us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":321,"threads_started":1}
W20260811 09:02:17.332553 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:17.323153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60253 (local address 127.9.40.126:39607)
0811 09:02:17.323391 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:17.323400 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:17.323718 (+ 318us) server_negotiation.cc:407] Connection header received
0811 09:02:17.323924 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:17.323931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:17.324002 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:17.324120 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:17.325012 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.325829 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.327128 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.327605 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.327756 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.332291 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:60253: BlockingRecv error: recv got EOF from 127.9.40.65:60253 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:17.367885 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:17.358402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:33271)
0811 09:02:17.358987 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:17.359004 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:17.359173 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:17.359517 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:17.359522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:17.360103 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:17.360289 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.360298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.361439 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.361450 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.362399 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.362411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.363195 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.363202 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.363394 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.364077 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:17.364097 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:17.367470 (+ 3373us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":293,"threads_started":1}
W20260811 09:02:17.368283 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:17.358474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33271 (local address 127.9.40.126:39607)
0811 09:02:17.358725 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:17.358736 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:17.359101 (+ 365us) server_negotiation.cc:407] Connection header received
0811 09:02:17.359299 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:17.359304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:17.359376 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:17.359483 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:17.360449 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.361276 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.362575 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.363012 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.363161 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.367782 (+ 4621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:33271: BlockingRecv error: recv got EOF from 127.9.40.67:33271 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:17 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:17.853264 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:17.843625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:56279)
0811 09:02:17.844264 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:17.844283 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:17.844462 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:17.844953 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:17.844958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:17.845520 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:17.845701 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.845711 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.846806 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.846811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.847767 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:17.847779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.848579 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.848584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:17.848751 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.849513 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:17.849532 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:17.852975 (+ 3443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":364,"threads_started":1}
W20260811 09:02:17.853487 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:17.843679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56279 (local address 127.9.40.126:39607)
0811 09:02:17.843927 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:17.843934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:17.844386 (+ 452us) server_negotiation.cc:407] Connection header received
0811 09:02:17.844623 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:17.844629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:17.844759 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:17.844867 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:17.845849 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.846637 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.847944 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:17.848405 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:17.848511 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:17.853242 (+ 4731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:56279: BlockingRecv error: recv got EOF from 127.9.40.66:56279 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:18.343555 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:18.333814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:35023)
0811 09:02:18.334557 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:18.334580 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:18.334736 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:18.335157 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:18.335162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:18.335775 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:18.335959 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.335968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.337221 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.337225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.338189 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.338202 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.339009 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.339014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.339138 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.339857 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:18.339877 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:18.343255 (+ 3378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":340,"threads_started":1}
W20260811 09:02:18.343856 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:18.333946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35023 (local address 127.9.40.126:39607)
0811 09:02:18.334161 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:18.334170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:18.334689 (+ 519us) server_negotiation.cc:407] Connection header received
0811 09:02:18.334886 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:18.334891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:18.334986 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:18.335085 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:18.336132 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.337070 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.338354 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.338839 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.338942 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.343572 (+ 4630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:35023: BlockingRecv error: recv got EOF from 127.9.40.65:35023 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:18.379381 20092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:18.369350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:46657)
0811 09:02:18.370007 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:18.370025 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:18.370356 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:18.370790 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:18.370795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:18.371255 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:18.371425 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.371434 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.372750 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.372756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.373844 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.373859 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.374671 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.374677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.374873 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.375603 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:18.375622 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:18.379141 (+ 3519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":349,"threads_started":1}
W20260811 09:02:18.379755 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:18.369584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46657 (local address 127.9.40.126:39607)
0811 09:02:18.369865 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:18.369871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:18.370248 (+ 377us) server_negotiation.cc:407] Connection header received
0811 09:02:18.370533 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:18.370539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:18.370632 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:18.370753 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:18.371603 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.372560 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.374007 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.374499 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.374648 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.379478 (+ 4830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:46657: BlockingRecv error: recv got EOF from 127.9.40.67:46657 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:18 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:18.871060 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:18.854397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:57913)
0811 09:02:18.855229 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:18.855276 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:18.855439 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:18.855893 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:18.855899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:18.856440 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:18.856638 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.856648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.857866 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.857870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.858893 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:18.858907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.859887 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.859895 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:18.860113 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.866689 (+ 6576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:18.866728 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:18.870793 (+ 4065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":305,"threads_started":1}
W20260811 09:02:18.871312 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:18.854924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57913 (local address 127.9.40.126:39607)
0811 09:02:18.855103 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:18.855108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:18.855401 (+ 293us) server_negotiation.cc:407] Connection header received
0811 09:02:18.855628 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:18.855633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:18.855719 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:18.855860 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:18.856873 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.857675 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.859070 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:18.859547 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:18.859658 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:18.871072 (+ 11414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:57913: BlockingRecv error: recv got EOF from 127.9.40.66:57913 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:19.355512 20094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:19.344932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:43949)
0811 09:02:19.345804 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:19.345829 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:19.346019 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:19.346422 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:19.346429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:19.346973 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:19.347139 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.347148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.348297 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.348306 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.349327 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.349342 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.350252 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.350260 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.350462 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.351234 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:19.351255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:19.355075 (+ 3820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":570,"threads_started":1}
W20260811 09:02:19.355666 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:19.345075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43949 (local address 127.9.40.126:39607)
0811 09:02:19.345318 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:19.345324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:19.345948 (+ 624us) server_negotiation.cc:407] Connection header received
0811 09:02:19.346163 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:19.346169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:19.346245 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:19.346370 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:19.347309 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.348110 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.349531 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.350022 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.350183 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.355382 (+ 5199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:43949: BlockingRecv error: recv got EOF from 127.9.40.65:43949 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:19.391268 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:19.380760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:54073)
0811 09:02:19.381458 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:19.381476 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:19.381621 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:19.382037 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:19.382040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:19.382496 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:19.382780 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.382802 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.384356 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.384363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.385577 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.385593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.386407 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.386412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.386549 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.387301 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:19.387321 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:19.390966 (+ 3645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":309,"threads_started":1}
W20260811 09:02:19.392055 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:19.381024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54073 (local address 127.9.40.126:39607)
0811 09:02:19.381269 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:19.381274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:19.381610 (+ 336us) server_negotiation.cc:407] Connection header received
0811 09:02:19.381760 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:19.381768 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:19.381864 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:19.381998 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:19.382998 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.384134 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.385747 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.386241 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.386348 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.391591 (+ 5243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:54073: BlockingRecv error: recv got EOF from 127.9.40.67:54073 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:19 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:19.882202 20096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:19.872445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:46773)
0811 09:02:19.873098 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:19.873122 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:19.873309 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:19.873663 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:19.873668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:19.874232 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:19.874402 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.874411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.875703 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.875709 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.876786 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:19.876799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.877702 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.877708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:19.877913 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.878589 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:19.878607 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:19.881841 (+ 3234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":562,"threads_started":1}
W20260811 09:02:19.882419 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:19.872558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46773 (local address 127.9.40.126:39607)
0811 09:02:19.872845 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:19.872854 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:19.873226 (+ 372us) server_negotiation.cc:407] Connection header received
0811 09:02:19.873436 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:19.873442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:19.873522 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:19.873629 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:19.874563 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.875542 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.876944 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:19.877463 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:19.877633 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:19.882157 (+ 4524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:46773: BlockingRecv error: recv got EOF from 127.9.40.66:46773 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:20.366055 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:20.356811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:35957)
0811 09:02:20.357372 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:20.357391 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:20.357515 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:20.357837 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:20.357840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:20.358305 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:20.358479 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.358488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.359549 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.359555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.360592 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.360607 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.361505 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.361510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.361655 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.362389 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:20.362408 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:20.365765 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":324,"threads_started":1}
W20260811 09:02:20.366485 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:20.356841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35957 (local address 127.9.40.126:39607)
0811 09:02:20.357085 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:20.357095 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:20.357479 (+ 384us) server_negotiation.cc:407] Connection header received
0811 09:02:20.357619 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:20.357623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:20.357704 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:20.357791 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:20.358613 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.359380 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.360827 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.361332 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.361459 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.366163 (+ 4704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:35957: BlockingRecv error: recv got EOF from 127.9.40.65:35957 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:20.402050 20099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:20.392627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:58651)
0811 09:02:20.393228 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:20.393245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:20.393382 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:20.393799 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:20.393803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:20.394298 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:20.394535 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.394546 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.395612 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.395618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.396567 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.396580 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.397484 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.397490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.397676 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.398368 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:20.398388 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:20.401680 (+ 3292us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":344,"threads_started":1}
W20260811 09:02:20.402257 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:20.392811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58651 (local address 127.9.40.126:39607)
0811 09:02:20.393072 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:20.393079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:20.393337 (+ 258us) server_negotiation.cc:407] Connection header received
0811 09:02:20.393581 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:20.393585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:20.393654 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:20.393766 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:20.394686 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.395456 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.396757 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.397269 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.397418 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.401970 (+ 4552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:58651: BlockingRecv error: recv got EOF from 127.9.40.67:58651 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:20 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:20.893698 20100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:20.883596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:55505)
0811 09:02:20.884225 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:20.884243 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:20.884426 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:20.884884 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:20.884889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:20.885371 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:20.885550 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.885559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.886802 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.886810 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.888145 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:20.888162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.889112 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.889119 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:20.889312 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.889940 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:20.889958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:20.893381 (+ 3423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":344,"threads_started":1}
W20260811 09:02:20.894017 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:20.883710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:55505 (local address 127.9.40.126:39607)
0811 09:02:20.884028 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:20.884035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:20.884343 (+ 308us) server_negotiation.cc:407] Connection header received
0811 09:02:20.884603 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:20.884607 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:20.884675 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:20.884850 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:20.885730 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.886654 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.888297 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:20.888934 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:20.889083 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:20.893709 (+ 4626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:55505: BlockingRecv error: recv got EOF from 127.9.40.66:55505 (error 108)
Metrics: {"server-negotiator.queue_time_us":113}
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:21.377092 20101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:21.367312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:49759)
0811 09:02:21.367906 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:21.367935 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:21.368112 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:21.368580 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:21.368586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:21.369232 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:21.369390 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.369400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.370581 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.370585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.371551 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.371586 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.372354 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.372359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.372502 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.373361 (+ 859us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:21.373381 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:21.376831 (+ 3450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":349,"threads_started":1}
W20260811 09:02:21.377477 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:21.367411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:49759 (local address 127.9.40.126:39607)
0811 09:02:21.367632 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:21.367652 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:21.368191 (+ 539us) server_negotiation.cc:407] Connection header received
0811 09:02:21.368278 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:21.368286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:21.368374 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:21.368474 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:21.369564 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.370403 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.371726 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.372200 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.372331 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.377187 (+ 4856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:49759: BlockingRecv error: recv got EOF from 127.9.40.65:49759 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:21.413165 20102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:21.403277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:56743)
0811 09:02:21.403879 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:21.403896 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:21.404071 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:21.404445 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:21.404449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:21.405083 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:21.405320 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.405334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.406730 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.406736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.407795 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.407810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.408645 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.408649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.408877 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.409551 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:21.409589 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:21.412889 (+ 3300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":339,"threads_started":1}
W20260811 09:02:21.413516 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:21.403434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56743 (local address 127.9.40.126:39607)
0811 09:02:21.403704 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:21.403713 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:21.403994 (+ 281us) server_negotiation.cc:407] Connection header received
0811 09:02:21.404210 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:21.404222 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:21.404290 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:21.404410 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:21.405511 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.406552 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.407959 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.408440 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.408554 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.413209 (+ 4655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:56743: BlockingRecv error: recv got EOF from 127.9.40.67:56743 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:21 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:21.904269 20103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:21.894884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:56271)
0811 09:02:21.895439 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:21.895456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:21.895633 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:21.896004 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:21.896022 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:21.896454 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:21.896612 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.896621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.897797 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.897803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.898799 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:21.898813 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.899638 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.899644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:21.899838 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.900545 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:21.900606 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:21.903999 (+ 3393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":323,"threads_started":1}
W20260811 09:02:21.904973 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:21.895003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56271 (local address 127.9.40.126:39607)
0811 09:02:21.895232 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:21.895237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:21.895565 (+ 328us) server_negotiation.cc:407] Connection header received
0811 09:02:21.895770 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:21.895775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:21.895852 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:21.895965 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:21.896811 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.897618 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.898968 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:21.899450 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:21.899614 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:21.904754 (+ 5140us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:56271: BlockingRecv error: recv got EOF from 127.9.40.66:56271 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:22.388520 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:22.378228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:39373)
0811 09:02:22.378832 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:22.378854 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:22.379042 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:22.379442 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:22.379446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:22.379967 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:22.380180 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.380190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.381395 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.381402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.382358 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.382372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.383180 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.383187 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.383372 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.384170 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:22.384191 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:22.388117 (+ 3926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":313,"threads_started":1}
W20260811 09:02:22.388731 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:22.378584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39373 (local address 127.9.40.126:39607)
0811 09:02:22.378828 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:22.378834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:22.378953 (+ 119us) server_negotiation.cc:407] Connection header received
0811 09:02:22.379200 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:22.379205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:22.379283 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:22.379401 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:22.380354 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.381231 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.382528 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.382987 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.383135 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.388357 (+ 5222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:39373: BlockingRecv error: recv got EOF from 127.9.40.65:39373 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:22.423461 20105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:22.414288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:51495)
0811 09:02:22.414856 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:22.414874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:22.415060 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:22.415428 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:22.415433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:22.415881 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:22.416042 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.416051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.417322 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.417328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.418366 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.418379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.419167 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.419174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.419363 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.420079 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:22.420098 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:22.423212 (+ 3114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":319,"threads_started":1}
W20260811 09:02:22.424031 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:22.414371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:51495 (local address 127.9.40.126:39607)
0811 09:02:22.414641 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:22.414649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:22.414974 (+ 325us) server_negotiation.cc:407] Connection header received
0811 09:02:22.415195 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:22.415201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:22.415278 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:22.415390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:22.416208 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.417180 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.418526 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.418985 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.419129 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.423499 (+ 4370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:51495: BlockingRecv error: recv got EOF from 127.9.40.67:51495 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:22 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:22.915428 20106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:22.905475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:45787)
0811 09:02:22.906092 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:22.906119 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:22.906258 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:22.906707 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:22.906712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:22.907179 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:22.907375 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.907384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.908463 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.908468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.909714 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:22.909730 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.910520 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.910525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:22.910651 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.911594 (+ 943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:22.911619 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:22.915149 (+ 3530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":360,"threads_started":1}
W20260811 09:02:22.915686 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:22.905697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:45787 (local address 127.9.40.126:39607)
0811 09:02:22.905907 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:22.905912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:22.906240 (+ 328us) server_negotiation.cc:407] Connection header received
0811 09:02:22.906375 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:22.906380 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:22.906466 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:22.906565 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:22.907534 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.908306 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.909878 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:22.910352 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:22.910459 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:22.915415 (+ 4956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:45787: BlockingRecv error: recv got EOF from 127.9.40.66:45787 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:23.399495 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:23.389830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:45481)
0811 09:02:23.390617 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:23.390633 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:23.390767 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:23.391525 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:23.391528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:23.391944 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:23.392120 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.392129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.393377 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.393381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.394381 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.394394 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.395173 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.395178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.395316 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.395964 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:23.395982 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:23.399235 (+ 3253us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":548,"threads_started":1}
W20260811 09:02:23.399821 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:23.389902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45481 (local address 127.9.40.126:39607)
0811 09:02:23.390176 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:23.390186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:23.390715 (+ 529us) server_negotiation.cc:407] Connection header received
0811 09:02:23.391054 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:23.391060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:23.391336 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:23.391462 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:23.392268 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.393225 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.394532 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.394980 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.395135 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.399501 (+ 4366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:45481: BlockingRecv error: recv got EOF from 127.9.40.65:45481 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:23.434165 20108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:23.424685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:33427)
0811 09:02:23.425266 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:23.425284 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:23.425416 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:23.425923 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:23.425931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:23.426615 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:23.426870 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.426881 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.428206 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.428210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.429227 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.429240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.429940 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.429944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.430071 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.430688 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:23.430706 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:23.433883 (+ 3177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":287,"threads_started":1}
W20260811 09:02:23.434386 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:23.424855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33427 (local address 127.9.40.126:39607)
0811 09:02:23.425156 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:23.425163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:23.425513 (+ 350us) server_negotiation.cc:407] Connection header received
0811 09:02:23.425638 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:23.425643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:23.425730 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:23.425842 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:23.427031 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.428076 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.429362 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.429800 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.429900 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.434141 (+ 4241us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:33427: BlockingRecv error: recv got EOF from 127.9.40.67:33427 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:23 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:23.929427 20109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:23.916496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:59597)
0811 09:02:23.917071 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:23.917090 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:23.917224 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:23.917631 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:23.917635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:23.918041 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:23.918209 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.918217 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.919359 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.919364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.920355 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:23.920369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.921198 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.921203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:23.921348 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.925267 (+ 3919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:23.925292 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:23.929165 (+ 3873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":347,"threads_started":1}
W20260811 09:02:23.929674 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:23.916571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59597 (local address 127.9.40.126:39607)
0811 09:02:23.916826 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:23.916831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:23.917178 (+ 347us) server_negotiation.cc:407] Connection header received
0811 09:02:23.917328 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:23.917332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:23.917418 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:23.917622 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:23.918379 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.919197 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.920516 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:23.921028 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:23.921154 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:23.929397 (+ 8243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:59597: BlockingRecv error: recv got EOF from 127.9.40.66:59597 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:24.411157 20110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:24.400881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:46105)
0811 09:02:24.401765 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:24.401791 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:24.402040 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:24.402744 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:24.402748 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:24.403272 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:24.403454 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.403464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.404571 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.404577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.405648 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.405661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.406608 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.406615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.406806 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.407503 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:24.407523 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:24.410872 (+ 3349us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":540,"threads_started":1}
W20260811 09:02:24.411504 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:24.401094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:46105 (local address 127.9.40.126:39607)
0811 09:02:24.401327 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:24.401336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:24.401900 (+ 564us) server_negotiation.cc:407] Connection header received
0811 09:02:24.402214 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:24.402220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:24.402306 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:24.402426 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:24.403634 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.404401 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.405830 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.406383 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.406555 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.411162 (+ 4607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:46105: BlockingRecv error: recv got EOF from 127.9.40.65:46105 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:24.445755 20111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:24.435529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:58535)
0811 09:02:24.436182 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:24.436199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:24.436386 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:24.436804 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:24.436809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:24.437296 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:24.437487 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.437496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.438783 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.438788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.440206 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.440227 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.441348 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.441353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.441494 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.442205 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:24.442225 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:24.445493 (+ 3268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":364,"threads_started":1}
W20260811 09:02:24.446098 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:24.435685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58535 (local address 127.9.40.126:39607)
0811 09:02:24.435972 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:24.435981 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:24.436316 (+ 335us) server_negotiation.cc:407] Connection header received
0811 09:02:24.436532 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:24.436537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:24.436630 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:24.436773 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:24.437670 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.438569 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.440411 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.441157 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.441261 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.445773 (+ 4512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:58535: BlockingRecv error: recv got EOF from 127.9.40.67:58535 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:24 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:24.947319 20112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:24.930564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:58893)
0811 09:02:24.931287 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:24.931317 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:24.931497 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:24.931881 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:24.931885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:24.932345 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:24.932514 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.932523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.933645 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.933651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.934645 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:24.934658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.935487 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.935494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:24.935684 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.943516 (+ 7832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:24.943545 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:24.947015 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":400,"threads_started":1}
W20260811 09:02:24.947768 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:24.930892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:58893 (local address 127.9.40.126:39607)
0811 09:02:24.931230 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:24.931239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:24.931410 (+ 171us) server_negotiation.cc:407] Connection header received
0811 09:02:24.931655 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:24.931660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:24.931728 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:24.931842 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:24.932724 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.933487 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.934813 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:24.935281 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:24.935435 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:24.947303 (+ 11868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:58893: BlockingRecv error: recv got EOF from 127.9.40.66:58893 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:25.423069 20113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:25.412455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:55591)
0811 09:02:25.413109 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:25.413128 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:25.413328 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:25.413760 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:25.413765 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:25.414235 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:25.414465 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.414476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.415953 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.415960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.417373 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.417390 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.418340 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.418348 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.418569 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.419285 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:25.419305 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:25.422757 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":378,"threads_started":1}
W20260811 09:02:25.423537 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:25.412626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:55591 (local address 127.9.40.126:39607)
0811 09:02:25.412967 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:25.412975 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:25.413236 (+ 261us) server_negotiation.cc:407] Connection header received
0811 09:02:25.413520 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:25.413524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:25.413600 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:25.413723 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:25.414647 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.415786 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.417558 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.418162 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.418323 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.423051 (+ 4728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:55591: BlockingRecv error: recv got EOF from 127.9.40.65:55591 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:25.456594 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:25.447195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:41429)
0811 09:02:25.447773 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:25.447790 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:25.447918 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:25.448270 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:25.448274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:25.448792 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:25.448971 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.448980 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.450021 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.450025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.450993 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.451006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.451725 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.451729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.451857 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.452539 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:25.452558 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:25.456190 (+ 3632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":308,"threads_started":1}
W20260811 09:02:25.456663 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:25.447303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41429 (local address 127.9.40.126:39607)
0811 09:02:25.447636 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:25.447664 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:25.447888 (+ 224us) server_negotiation.cc:407] Connection header received
0811 09:02:25.448049 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:25.448053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:25.448121 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:25.448220 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:25.449119 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.449878 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.451157 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.451583 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.451688 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.456431 (+ 4743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:41429: BlockingRecv error: recv got EOF from 127.9.40.67:41429 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:25 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:25.957933 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:25.948553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:52747)
0811 09:02:25.949188 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:25.949217 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:25.949349 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:25.949681 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:25.949685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:25.950102 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:25.950321 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.950331 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.951438 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.951443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.952455 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:25.952469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.953310 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.953315 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:25.953451 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.954137 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:25.954155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:25.957587 (+ 3432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":370,"threads_started":1}
W20260811 09:02:25.958103 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:25.948823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52747 (local address 127.9.40.126:39607)
0811 09:02:25.949051 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:25.949056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:25.949303 (+ 247us) server_negotiation.cc:407] Connection header received
0811 09:02:25.949457 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:25.949462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:25.949532 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:25.949636 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:25.950473 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.951274 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.952620 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:25.953159 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:25.953276 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:25.957811 (+ 4535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:52747: BlockingRecv error: recv got EOF from 127.9.40.66:52747 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:26.434580 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:26.424487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.65:44141)
0811 09:02:26.425198 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:26.425215 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:26.425400 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:26.425780 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:26.425785 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:26.426301 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:26.426519 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.426530 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.427617 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.427623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.428616 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.428628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.429567 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.429576 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.429765 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.430546 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:26.430572 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:26.434298 (+ 3726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":400,"threads_started":1}
W20260811 09:02:26.434805 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:26.424767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44141 (local address 127.9.40.126:39607)
0811 09:02:26.425039 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:26.425046 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:26.425318 (+ 272us) server_negotiation.cc:407] Connection header received
0811 09:02:26.425545 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:26.425551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:26.425630 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:26.425746 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:26.426693 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.427451 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.428854 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.429372 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.429536 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.434569 (+ 5033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:44141: BlockingRecv error: recv got EOF from 127.9.40.65:44141 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:26.466676 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:26.457811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.67:55395)
0811 09:02:26.458349 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:26.458369 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:26.458528 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:26.458914 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:26.458919 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:26.459334 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:26.459526 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.459536 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.460638 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.460644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.461636 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.461648 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.462445 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.462451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.462649 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.463279 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:26.463304 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:26.466396 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":282,"threads_started":1}
W20260811 09:02:26.466957 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:26.457968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:55395 (local address 127.9.40.126:39607)
0811 09:02:26.458143 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:26.458148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:26.458450 (+ 302us) server_negotiation.cc:407] Connection header received
0811 09:02:26.458691 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:26.458697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:26.458777 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:26.458883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:26.459700 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.460463 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.461802 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.462278 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.462425 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.466687 (+ 4262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:55395: BlockingRecv error: recv got EOF from 127.9.40.67:55395 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:02:26 dist-test-slave-cbsf krb5kdc[18933](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:02:26.969241 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:26.959328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39607 (local address 127.9.40.66:46997)
0811 09:02:26.959874 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:26.959891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:26.960065 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:26.960442 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:26.960447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:26.961011 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:26.961235 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.961244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.962355 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.962361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.963336 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:26.963350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.964176 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.964182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:26.964368 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.965143 (+ 775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:26.965161 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:26.968768 (+ 3607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:39607: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":306,"threads_started":1}
W20260811 09:02:26.969451 20068 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:26.959402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46997 (local address 127.9.40.126:39607)
0811 09:02:26.959688 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:26.959698 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:26.959980 (+ 282us) server_negotiation.cc:407] Connection header received
0811 09:02:26.960225 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:26.960229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:26.960296 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:26.960410 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:26.961400 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.962186 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.963512 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:26.963994 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:26.964154 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:26.969153 (+ 4999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:46997: BlockingRecv error: recv got EOF from 127.9.40.66:46997 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:02:27.277842 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19583
I20260811 09:02:27.291612 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:27.279196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44926 (local address 127.9.40.65:38977)
0811 09:02:27.279902 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:27.279911 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:27.279941 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:02:27.280008 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:27.280015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:27.280259 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:27.280420 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:27.282108 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.283235 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.284562 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.285267 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.285380 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:27.288139 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:27.288159 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:27.288176 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:27.288215 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:27.291006 (+ 2791us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:02:27.291332 (+ 326us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44926: Request ticket server kudu/127.9.40.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":383,"threads_started":1}
W20260811 09:02:27.291739 20120 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44926: Request ticket server kudu/127.9.40.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:02:27.291906 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19583: Not authorized: Client connection negotiation failed: client connection to 127.9.40.65:38977: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:02:27.292194 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19583
I20260811 09:02:27.316630 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19757
I20260811 09:02:27.331235 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:27.318014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34198 (local address 127.9.40.66:46433)
0811 09:02:27.318718 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:27.318726 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:27.318760 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:02:27.318871 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:27.318878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:27.319118 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:27.319287 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:27.320782 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.321909 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.323285 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.323951 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.324142 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:27.327729 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:27.327751 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:27.327771 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:27.327816 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:27.330628 (+ 2812us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:02:27.330957 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34198: Request ticket server kudu/127.9.40.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":345,"threads_started":1}
W20260811 09:02:27.331393 20121 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34198: Request ticket server kudu/127.9.40.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:02:27.331566 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19757: Not authorized: Client connection negotiation failed: client connection to 127.9.40.66:46433: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:02:27.331724 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19757
I20260811 09:02:27.356058 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19918
I20260811 09:02:27.370324 20122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:27.357175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.9.40.67:39421)
0811 09:02:27.357822 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:27.357830 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:27.357859 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:02:27.357925 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:27.357932 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:27.358185 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:27.358344 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:27.359914 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.361271 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.362602 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:27.363256 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:27.363368 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:27.366476 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:27.366488 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:27.366507 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:27.366569 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:27.369663 (+ 3094us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:02:27.370025 (+ 362us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54420: Request ticket server kudu/127.9.40.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":333,"threads_started":1}
W20260811 09:02:27.370462 20122 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54420: Request ticket server kudu/127.9.40.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:02:27.370546 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19918: Not authorized: Client connection negotiation failed: client connection to 127.9.40.67:39421: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.9.40.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:02:27.370807 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19918
I20260811 09:02:27.392943 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 19512
I20260811 09:02:27.394369 19576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:27.520660 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 19512
2026-08-11T09:02:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22257 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T09:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:27Z Disabled control of system clock
I20260811 09:02:27.640343 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39929
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39929
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:02:28.012970 20128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:28.013321 20128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:28.013460 20128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:28.021461 20128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:28.021612 20128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:28.021651 20128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:28.021770 20128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:28.021834 20128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:28.033455 20128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:28.035079 20128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:28.037367 20128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:28.052443 20137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:28.052454 20133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:28.052320 20134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:28.053439 20128 server_base.cc:1034] running on GCE node
I20260811 09:02:28.054412 20128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:28.056244 20128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:28.057561 20128 hybrid_clock.cc:648] HybridClock initialized: now 1786438948057507 us; error 73 us; skew 500 ppm
I20260811 09:02:28.058033 20128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:28.061264 20128 webserver.cc:460] Webserver started at http://127.9.40.126:43039/ using document root <none> and password file <none>
I20260811 09:02:28.062214 20128 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:28.062381 20128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:28.062788 20128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:28.066511 20128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "e00a0403ab0648cbb6edf1da2b3f13f2"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "ab5f13fb73ff393514c3a52beddbb0f5"
server_key_iv: "6c75a2580f04b86346bc71597d0fa162"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.067538 20128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "e00a0403ab0648cbb6edf1da2b3f13f2"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "ab5f13fb73ff393514c3a52beddbb0f5"
server_key_iv: "6c75a2580f04b86346bc71597d0fa162"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.074347 20128 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:02:28.079063 20143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:28.080921 20128 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:28.081214 20128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "e00a0403ab0648cbb6edf1da2b3f13f2"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "ab5f13fb73ff393514c3a52beddbb0f5"
server_key_iv: "6c75a2580f04b86346bc71597d0fa162"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.081503 20128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:28.113070 20128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:28.114092 20128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:28.114462 20128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:28.136335 20128 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39929
I20260811 09:02:28.136354 20202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39929 every 8 connection(s)
I20260811 09:02:28.138221 20128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:02:28.142895 20203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:28.145301 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20128
I20260811 09:02:28.145553 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:02:28.146306 9377 external_mini_cluster.cc:1428] Setting key 817539d159d5131f3ee98f01c7f19adf
I20260811 09:02:28.159766 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2: Bootstrap starting.
I20260811 09:02:28.164407 20203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:28.165591 20206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:28.148622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41318 (local address 127.9.40.126:39929)
0811 09:02:28.150126 (+ 1504us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:28.150158 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:28.150229 (+ 71us) server_negotiation.cc:407] Connection header received
0811 09:02:28.152926 (+ 2697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:28.152967 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:28.153736 (+ 769us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:28.154288 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:28.154962 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.157286 (+ 2324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.159100 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.160307 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.160519 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:28.162104 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:28.162119 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:28.162142 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:28.162966 (+ 824us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:28.163108 (+ 142us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:28.163145 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:28.163310 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:28.163545 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 09:02:28.163955 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":526,"threads_started":1}
I20260811 09:02:28.166056 20203 log.cc:824] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:28.170140 20203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2: No bootstrap required, opened a new log
I20260811 09:02:28.179114 20203 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } }
I20260811 09:02:28.179621 20203 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:28.179745 20203 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00a0403ab0648cbb6edf1da2b3f13f2, State: Initialized, Role: FOLLOWER
I20260811 09:02:28.180500 20203 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } }
I20260811 09:02:28.180817 20203 raft_consensus.cc:388] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:02:28.180986 20203 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:02:28.181227 20203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:28.184300 20203 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } }
I20260811 09:02:28.185031 20203 leader_election.cc:302] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e00a0403ab0648cbb6edf1da2b3f13f2; no voters:
I20260811 09:02:28.185842 20203 leader_election.cc:288] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:02:28.186012 20208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:28.187321 20208 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [term 1 LEADER]: Becoming Leader. State: Replica: e00a0403ab0648cbb6edf1da2b3f13f2, State: Running, Role: LEADER
I20260811 09:02:28.188045 20208 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } }
I20260811 09:02:28.188512 20203 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:02:28.191167 20209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } } }
I20260811 09:02:28.191566 20209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:28.192591 20213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:02:28.192410 20210 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00a0403ab0648cbb6edf1da2b3f13f2. Latest consensus state: current_term: 1 leader_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00a0403ab0648cbb6edf1da2b3f13f2" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39929 } } }
I20260811 09:02:28.192924 20210 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:28.199558 20213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:02:28.210069 20213 catalog_manager.cc:1347] Generated new cluster ID: 672d414819814a0b9c1ae54caee2cc4b
I20260811 09:02:28.210228 20213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:02:28.221383 20213 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:02:28.223523 20213 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:02:28.234079 20213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e00a0403ab0648cbb6edf1da2b3f13f2: Generated new TSK 0
I20260811 09:02:28.235399 20213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:02:28.243827 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--builtin_ntp_servers=127.9.40.84:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 09:02:28.645195 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:28.645527 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:28.645694 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:28.653579 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:28.653720 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:28.653919 20227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:02:28.664266 20227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:28.665921 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:28.668164 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:28.683745 20232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:28.683797 20234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:28.683833 20236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:28.684963 20227 server_base.cc:1034] running on GCE node
I20260811 09:02:28.685755 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:28.687011 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:28.688262 20227 hybrid_clock.cc:648] HybridClock initialized: now 1786438948688184 us; error 74 us; skew 500 ppm
I20260811 09:02:28.688779 20227 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:28.691790 20227 webserver.cc:460] Webserver started at http://127.9.40.65:41457/ using document root <none> and password file <none>
I20260811 09:02:28.692584 20227 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:28.692749 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:28.693120 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:28.696946 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "9ca26a8715cf4f829b940956edc668b5"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "49c3e324632cb569c026a897c283ae02"
server_key_iv: "0a0a0975d6fbd77ce24b4def71755a26"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.697970 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "9ca26a8715cf4f829b940956edc668b5"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "49c3e324632cb569c026a897c283ae02"
server_key_iv: "0a0a0975d6fbd77ce24b4def71755a26"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.704813 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 09:02:28.709208 20243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:28.711544 20227 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:02:28.711804 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "9ca26a8715cf4f829b940956edc668b5"
format_stamp: "Formatted at 2026-08-11 09:02:28 on dist-test-slave-cbsf"
server_key: "49c3e324632cb569c026a897c283ae02"
server_key_iv: "0a0a0975d6fbd77ce24b4def71755a26"
server_key_version: "encryptionkey@0"
I20260811 09:02:28.712158 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:28.730494 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:28.731561 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:28.731925 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:28.733180 20227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:28.735116 20227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:28.735272 20227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:28.735425 20227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:28.735520 20227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:28.773559 20227 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45471
I20260811 09:02:28.773617 20369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45471 every 8 connection(s)
I20260811 09:02:28.775084 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:02:28.778560 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20227
I20260811 09:02:28.778800 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:02:28.779562 9377 external_mini_cluster.cc:1428] Setting key 63e9c90e49069f43ea0c82bde8a98428
I20260811 09:02:28.782569 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--builtin_ntp_servers=127.9.40.84:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:02:28.790356 20372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:28.778187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36689 (local address 127.9.40.126:39929)
0811 09:02:28.778845 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:28.778852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:28.780313 (+ 1461us) server_negotiation.cc:407] Connection header received
0811 09:02:28.782376 (+ 2063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:28.782382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:28.782485 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:28.782614 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:28.783947 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.784809 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.786809 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.787255 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.787425 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:28.789289 (+ 1864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:28.789295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:28.789298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:28.789321 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:28.789370 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:28.789384 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:28.789502 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:28.789781 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 09:02:28.789954 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":333,"threads_started":1}
I20260811 09:02:28.791600 20373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:28.778856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39929 (local address 127.9.40.65:36689)
0811 09:02:28.780074 (+ 1218us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:28.780145 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:28.782033 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:28.782931 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:28.782940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:28.783051 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:28.783738 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:28.783758 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.785000 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.785005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:28.786611 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:28.786629 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:28.787551 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:28.787559 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:28.787838 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:28.788810 (+ 972us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:28.788813 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:28.788901 (+ 88us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:28.789009 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:28.789518 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:28.789535 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:28.789549 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 09:02:28.789766 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 09:02:28.790297 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":389,"threads_started":1}
I20260811 09:02:28.793792 20370 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39929
I20260811 09:02:28.794198 20370 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:28.795426 20370 heartbeater.cc:511] Master 127.9.40.126:39929 requested a full tablet report, sending...
I20260811 09:02:28.798293 20164 ts_manager.cc:194] Registered new tserver with Master: 9ca26a8715cf4f829b940956edc668b5 (127.9.40.65:45471)
I20260811 09:02:28.799994 20164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:36689
W20260811 09:02:29.176407 20374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:29.176864 20374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:29.177026 20374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:29.185623 20374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:29.185786 20374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:29.185961 20374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:02:29.196559 20374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:29.198230 20374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:29.200301 20374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:29.215101 20379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:29.215023 20380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:29.215780 20383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:29.216571 20374 server_base.cc:1034] running on GCE node
I20260811 09:02:29.217551 20374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:29.218837 20374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:29.220146 20374 hybrid_clock.cc:648] HybridClock initialized: now 1786438949220144 us; error 104 us; skew 500 ppm
I20260811 09:02:29.220672 20374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:29.227206 20374 webserver.cc:460] Webserver started at http://127.9.40.66:42701/ using document root <none> and password file <none>
I20260811 09:02:29.228152 20374 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:29.228331 20374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:29.228860 20374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:29.232239 20374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "5ee0159424a74d1ba8b227f806438f38"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f93db5bb30bc69c475602d5805d3925f"
server_key_iv: "0b65e918c39f7a70bc5c37a030b717c0"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.233268 20374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "5ee0159424a74d1ba8b227f806438f38"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f93db5bb30bc69c475602d5805d3925f"
server_key_iv: "0b65e918c39f7a70bc5c37a030b717c0"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.239655 20374 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260811 09:02:29.243993 20389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.245797 20374 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:29.246048 20374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "5ee0159424a74d1ba8b227f806438f38"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f93db5bb30bc69c475602d5805d3925f"
server_key_iv: "0b65e918c39f7a70bc5c37a030b717c0"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.246344 20374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:29.276489 20374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:29.277662 20374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:29.278000 20374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:29.279173 20374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:29.281193 20374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:29.281334 20374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.281499 20374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:29.281611 20374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.320276 20374 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35607
I20260811 09:02:29.320379 20510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35607 every 8 connection(s)
I20260811 09:02:29.321928 20374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:02:29.331297 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20374
I20260811 09:02:29.331456 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:02:29.332001 9377 external_mini_cluster.cc:1428] Setting key d3179f911a9643ee5f4a07722ff9b875
I20260811 09:02:29.335413 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--builtin_ntp_servers=127.9.40.84:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:02:29.338331 20513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:29.325045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:34205 (local address 127.9.40.126:39929)
0811 09:02:29.325642 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:29.325647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:29.326770 (+ 1123us) server_negotiation.cc:407] Connection header received
0811 09:02:29.329165 (+ 2395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:29.329171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:29.329291 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:29.329437 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:29.330853 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.331886 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.334674 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.335165 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.335341 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:29.337278 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:29.337284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:29.337288 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:29.337311 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:29.337361 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:29.337375 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:29.337491 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:29.337786 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 09:02:29.337916 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":338,"threads_started":1}
I20260811 09:02:29.339622 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:29.325448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39929 (local address 127.9.40.66:34205)
0811 09:02:29.326558 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:29.326603 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:29.328690 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:29.329749 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:29.329760 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:29.329886 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:29.330607 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:29.330626 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.332142 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.332149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:29.334471 (+ 2322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:29.334493 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.335472 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.335482 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:29.335862 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:29.336809 (+ 947us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:29.336812 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:29.336866 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:29.336979 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:29.337493 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:29.337509 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:29.337522 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:02:29.337760 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 09:02:29.338294 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":348,"threads_started":1}
I20260811 09:02:29.342017 20511 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39929
I20260811 09:02:29.342679 20511 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:29.344089 20511 heartbeater.cc:511] Master 127.9.40.126:39929 requested a full tablet report, sending...
I20260811 09:02:29.346364 20164 ts_manager.cc:194] Registered new tserver with Master: 5ee0159424a74d1ba8b227f806438f38 (127.9.40.66:35607)
I20260811 09:02:29.347568 20164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:34205
W20260811 09:02:29.724002 20515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:29.724421 20515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:29.724578 20515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:29.733850 20515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:29.734018 20515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:29.734259 20515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:02:29.748245 20515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39929
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:29.749799 20515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:29.751967 20515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:29.767397 20521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:29.767510 20523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:29.767447 20520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:29.768733 20515 server_base.cc:1034] running on GCE node
I20260811 09:02:29.769586 20515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:29.770799 20515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:29.772092 20515 hybrid_clock.cc:648] HybridClock initialized: now 1786438949772016 us; error 78 us; skew 500 ppm
I20260811 09:02:29.772482 20515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:29.775488 20515 webserver.cc:460] Webserver started at http://127.9.40.67:42387/ using document root <none> and password file <none>
I20260811 09:02:29.776338 20515 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:29.776495 20515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:29.776922 20515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:29.783093 20515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "ef44756d6d5e4ad4b614f3411771d753"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f75846eda8c10587f17ba95b474367f7"
server_key_iv: "22432e07fd84d0246a8214e27d92b457"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.784096 20515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "ef44756d6d5e4ad4b614f3411771d753"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f75846eda8c10587f17ba95b474367f7"
server_key_iv: "22432e07fd84d0246a8214e27d92b457"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.791746 20515 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 09:02:29.796918 20530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.799048 20515 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:02:29.799335 20515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "ef44756d6d5e4ad4b614f3411771d753"
format_stamp: "Formatted at 2026-08-11 09:02:29 on dist-test-slave-cbsf"
server_key: "f75846eda8c10587f17ba95b474367f7"
server_key_iv: "22432e07fd84d0246a8214e27d92b457"
server_key_version: "encryptionkey@0"
I20260811 09:02:29.799633 20515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:29.804970 20370 heartbeater.cc:503] Master 127.9.40.126:39929 was elected leader, sending a full tablet report...
I20260811 09:02:29.825356 20515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:29.826421 20515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:29.826701 20515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:29.827833 20515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:29.829609 20515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:29.829710 20515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.829790 20515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:29.829849 20515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:29.867147 20515 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44001
I20260811 09:02:29.867256 20659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44001 every 8 connection(s)
I20260811 09:02:29.868655 20515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:02:29.872684 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20515
I20260811 09:02:29.872936 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:02:29.873596 9377 external_mini_cluster.cc:1428] Setting key dd726cc782eb2faddb5183716d694ddd
I20260811 09:02:29.884398 20662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:29.871844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33801 (local address 127.9.40.126:39929)
0811 09:02:29.872350 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:29.872355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:29.873975 (+ 1620us) server_negotiation.cc:407] Connection header received
0811 09:02:29.876600 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:29.876605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:29.876683 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:29.876858 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:29.878291 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.879122 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.881105 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.881560 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.881670 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:29.883483 (+ 1813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:29.883493 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:29.883498 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:29.883528 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:29.883588 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:29.883606 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:29.883719 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:29.883999 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 09:02:29.884105 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":265,"threads_started":1}
I20260811 09:02:29.885721 20663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:29.872466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39929 (local address 127.9.40.67:33801)
0811 09:02:29.873780 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:29.873828 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:29.876203 (+ 2375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:29.877231 (+ 1028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:29.877246 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:29.877395 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:29.878109 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:29.878129 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.879307 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.879313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:29.880928 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:29.880944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.881801 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.881808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:29.882054 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:29.883043 (+ 989us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:29.883046 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:29.883095 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:29.883204 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:29.883746 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:29.883757 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:29.883765 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:02:29.883960 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 09:02:29.884357 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":472,"threads_started":1}
I20260811 09:02:29.887641 20660 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39929
I20260811 09:02:29.888041 20660 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:29.889117 20660 heartbeater.cc:511] Master 127.9.40.126:39929 requested a full tablet report, sending...
I20260811 09:02:29.890923 20164 ts_manager.cc:194] Registered new tserver with Master: ef44756d6d5e4ad4b614f3411771d753 (127.9.40.67:44001)
I20260811 09:02:29.891809 20164 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:33801
I20260811 09:02:29.904676 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:02:29.912287 20662 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:29.910502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41326 (local address 127.9.40.126:39929)
0811 09:02:29.910916 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:29.910921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:29.911186 (+ 265us) server_negotiation.cc:407] Connection header received
0811 09:02:29.911338 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:29.911342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:29.911422 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:29.911544 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:29.911959 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41326: BlockingRecv error: recv got EOF from 127.0.0.1:41326 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
I20260811 09:02:29.914337 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20227
I20260811 09:02:29.923914 20670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:29.915443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41002 (local address 127.9.40.65:45471)
0811 09:02:29.916037 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:29.916045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:29.916088 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:02:29.916159 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:29.916166 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:29.916411 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:29.916559 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:29.917108 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.918265 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.919854 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:29.920550 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:29.920674 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:29.922088 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:29.922099 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:29.922118 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:29.922871 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:29.922931 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:29.922955 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:29.923066 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:29.923298 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 09:02:29.923488 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":269,"threads_started":1}
I20260811 09:02:29.925879 20364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:30.015851 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20227
I20260811 09:02:30.039290 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20374
I20260811 09:02:30.048393 20672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:30.040369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55594 (local address 127.9.40.66:35607)
0811 09:02:30.041042 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:30.041051 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:30.041082 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:02:30.041155 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:30.041162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:30.041407 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:30.041555 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:30.042054 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:30.043233 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:30.044658 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:30.045523 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:30.045660 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:30.046812 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:30.046822 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:30.046838 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:30.047535 (+ 697us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:30.047591 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:30.047614 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:30.047758 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:30.047927 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 09:02:30.048063 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":383,"threads_started":1}
I20260811 09:02:30.050040 20506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:30.138175 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20374
I20260811 09:02:30.162279 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20515
I20260811 09:02:30.171739 20674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:30.163422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39180 (local address 127.9.40.67:44001)
0811 09:02:30.164099 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:30.164107 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:30.164143 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:02:30.164209 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:30.164216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:30.164456 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:30.164665 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:30.165244 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:30.166429 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:30.167811 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:30.168489 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:30.168673 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:30.169969 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:30.169979 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:30.169996 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:30.170685 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:30.170741 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:30.170763 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:30.170914 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:30.171116 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 09:02:30.171292 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":355,"threads_started":1}
I20260811 09:02:30.173454 20655 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:30.260406 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20515
I20260811 09:02:30.283659 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20128
I20260811 09:02:30.285499 20198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:30.396431 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20128
2026-08-11T09:02:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2868 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T09:02:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:30Z Disabled control of system clock
I20260811 09:02:30.495322 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36227
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36227
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:02:30.883809 20681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:30.884188 20681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:30.884280 20681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:30.893172 20681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:30.893357 20681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:30.893404 20681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:30.893482 20681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:30.893532 20681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:30.905002 20681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36227
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:30.906512 20681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:30.908723 20681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:30.923846 20687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:30.923869 20688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:30.923844 20690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:30.925642 20681 server_base.cc:1034] running on GCE node
I20260811 09:02:30.926618 20681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:30.928362 20681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:30.929662 20681 hybrid_clock.cc:648] HybridClock initialized: now 1786438950929576 us; error 92 us; skew 500 ppm
I20260811 09:02:30.930140 20681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:30.933456 20681 webserver.cc:460] Webserver started at http://127.9.40.126:36333/ using document root <none> and password file <none>
I20260811 09:02:30.934398 20681 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:30.934566 20681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:30.934947 20681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:30.938529 20681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "2b25f839ddea4d2085b6d350811d51f8"
format_stamp: "Formatted at 2026-08-11 09:02:30 on dist-test-slave-cbsf"
server_key: "3f2bfd58742af86a8a385924cab0c0e5"
server_key_iv: "51008c00836b58801094c48ef2cbe4c5"
server_key_version: "encryptionkey@0"
I20260811 09:02:30.939558 20681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2b25f839ddea4d2085b6d350811d51f8"
format_stamp: "Formatted at 2026-08-11 09:02:30 on dist-test-slave-cbsf"
server_key: "3f2bfd58742af86a8a385924cab0c0e5"
server_key_iv: "51008c00836b58801094c48ef2cbe4c5"
server_key_version: "encryptionkey@0"
I20260811 09:02:30.946156 20681 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:02:30.950996 20697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:30.952848 20681 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 09:02:30.953114 20681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2b25f839ddea4d2085b6d350811d51f8"
format_stamp: "Formatted at 2026-08-11 09:02:30 on dist-test-slave-cbsf"
server_key: "3f2bfd58742af86a8a385924cab0c0e5"
server_key_iv: "51008c00836b58801094c48ef2cbe4c5"
server_key_version: "encryptionkey@0"
I20260811 09:02:30.953406 20681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:30.961169 20681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:30.962236 20681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:30.962623 20681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:30.984375 20681 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36227
I20260811 09:02:30.984393 20755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36227 every 8 connection(s)
I20260811 09:02:30.986063 20681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:02:30.990758 20756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:30.990803 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20681
I20260811 09:02:30.991060 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:02:30.992074 9377 external_mini_cluster.cc:1428] Setting key 1501d7725e00d240a012730ee09aeacf
I20260811 09:02:31.004791 20759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:30.996175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.9.40.126:36227)
0811 09:02:30.998086 (+ 1911us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:30.998104 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:30.998170 (+ 66us) server_negotiation.cc:407] Connection header received
0811 09:02:31.000165 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:31.000347 (+ 182us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:31.001029 (+ 682us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:31.001541 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:31.002778 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:31.002786 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:31.002800 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:31.002817 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:31.002886 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:31.002904 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:31.003171 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:31.003321 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 09:02:31.003627 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1200,"thread_start_us":817,"threads_started":1}
I20260811 09:02:31.007972 20756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8: Bootstrap starting.
I20260811 09:02:31.011751 20756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:31.013186 20756 log.cc:824] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:31.016803 20756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8: No bootstrap required, opened a new log
I20260811 09:02:31.025427 20756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } }
I20260811 09:02:31.025936 20756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:31.026059 20756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b25f839ddea4d2085b6d350811d51f8, State: Initialized, Role: FOLLOWER
I20260811 09:02:31.026767 20756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } }
I20260811 09:02:31.027069 20756 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:02:31.027240 20756 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:02:31.027451 20756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:31.030254 20756 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } }
I20260811 09:02:31.030915 20756 leader_election.cc:302] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b25f839ddea4d2085b6d350811d51f8; no voters:
I20260811 09:02:31.031708 20756 leader_election.cc:288] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:02:31.031903 20761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:31.032779 20761 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [term 1 LEADER]: Becoming Leader. State: Replica: 2b25f839ddea4d2085b6d350811d51f8, State: Running, Role: LEADER
I20260811 09:02:31.033435 20761 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } }
I20260811 09:02:31.034155 20756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:02:31.037063 20763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b25f839ddea4d2085b6d350811d51f8. Latest consensus state: current_term: 1 leader_uuid: "2b25f839ddea4d2085b6d350811d51f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } } }
I20260811 09:02:31.037995 20763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:31.038244 20762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b25f839ddea4d2085b6d350811d51f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b25f839ddea4d2085b6d350811d51f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36227 } } }
I20260811 09:02:31.038668 20762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:31.040773 20770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:02:31.047420 20770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:02:31.059053 20770 catalog_manager.cc:1347] Generated new cluster ID: 058196f1681946c1a108a6315ebe9d63
I20260811 09:02:31.059211 20770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:02:31.083936 20770 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:02:31.084506 20770 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:02:31.094597 20770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b25f839ddea4d2085b6d350811d51f8: Generated new TSK 0
I20260811 09:02:31.095908 20770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:02:31.112500 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--builtin_ntp_servers=127.9.40.84:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:02:31.526273 20787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:31.526639 20787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:31.526810 20787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:31.535019 20787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:31.535210 20787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:31.535481 20787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:02:31.546248 20787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:31.547816 20787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:31.550093 20787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:31.565948 20796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:31.565941 20793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:31.566684 20794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:31.566921 20787 server_base.cc:1034] running on GCE node
I20260811 09:02:31.567785 20787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:31.569059 20787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:31.570346 20787 hybrid_clock.cc:648] HybridClock initialized: now 1786438951570261 us; error 70 us; skew 500 ppm
I20260811 09:02:31.570832 20787 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:31.573769 20787 webserver.cc:460] Webserver started at http://127.9.40.65:45709/ using document root <none> and password file <none>
I20260811 09:02:31.574640 20787 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:31.574774 20787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:31.575171 20787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:31.578708 20787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "5a5225d429fe4c6a9f04e073ad356029"
format_stamp: "Formatted at 2026-08-11 09:02:31 on dist-test-slave-cbsf"
server_key: "6d10e503d42a9bd6413afb3291cf6b9d"
server_key_iv: "342ee8f4892d4a65bfc54a2a73fa04e0"
server_key_version: "encryptionkey@0"
I20260811 09:02:31.579692 20787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "5a5225d429fe4c6a9f04e073ad356029"
format_stamp: "Formatted at 2026-08-11 09:02:31 on dist-test-slave-cbsf"
server_key: "6d10e503d42a9bd6413afb3291cf6b9d"
server_key_iv: "342ee8f4892d4a65bfc54a2a73fa04e0"
server_key_version: "encryptionkey@0"
I20260811 09:02:31.586056 20787 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:02:31.590808 20802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:31.592546 20787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 09:02:31.592855 20787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "5a5225d429fe4c6a9f04e073ad356029"
format_stamp: "Formatted at 2026-08-11 09:02:31 on dist-test-slave-cbsf"
server_key: "6d10e503d42a9bd6413afb3291cf6b9d"
server_key_iv: "342ee8f4892d4a65bfc54a2a73fa04e0"
server_key_version: "encryptionkey@0"
I20260811 09:02:31.593154 20787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:31.600289 20787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:31.601397 20787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:31.601730 20787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:31.602890 20787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:31.604713 20787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:31.604867 20787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 09:02:31.605013 20787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:31.605118 20787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:31.644311 20787 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:35653
I20260811 09:02:31.644387 20927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:35653 every 8 connection(s)
I20260811 09:02:31.645890 20787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:02:31.650074 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20787
I20260811 09:02:31.650369 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:02:31.651269 9377 external_mini_cluster.cc:1428] Setting key 473acf29fe00b1fc6b10d118bbe541b7
I20260811 09:02:31.654659 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--builtin_ntp_servers=127.9.40.84:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:02:31.656873 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:31.649049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56233 (local address 127.9.40.126:36227)
0811 09:02:31.650393 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:31.650399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:31.651459 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 09:02:31.653850 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:31.653856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:31.653946 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:31.654087 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:31.655914 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:31.655920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:31.655923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:31.655944 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:31.655995 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:31.656008 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:31.656167 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:31.656401 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 09:02:31.656482 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":324,"threads_started":1}
I20260811 09:02:31.658120 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:31.649873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36227 (local address 127.9.40.65:56233)
0811 09:02:31.651238 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:31.651294 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:31.653487 (+ 2193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:31.654375 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:31.654386 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:31.654505 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:31.655487 (+ 982us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:31.655490 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:31.655543 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:31.655665 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:31.656126 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:31.656140 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:31.656158 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 09:02:31.656521 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 09:02:31.656915 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":574,"threads_started":1}
I20260811 09:02:31.660331 20928 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36227
I20260811 09:02:31.660778 20928 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:31.661557 20928 heartbeater.cc:511] Master 127.9.40.126:36227 requested a full tablet report, sending...
I20260811 09:02:31.664795 20716 ts_manager.cc:194] Registered new tserver with Master: 5a5225d429fe4c6a9f04e073ad356029 (127.9.40.65:35653)
W20260811 09:02:32.050014 20934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:32.050351 20934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:32.050449 20934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:32.058364 20934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:32.058498 20934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:32.058615 20934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:02:32.068830 20934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:32.070377 20934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:32.072587 20934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:32.088032 20944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:32.088061 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:32.088150 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:32.094739 20934 server_base.cc:1034] running on GCE node
I20260811 09:02:32.095666 20934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:32.096997 20934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:32.098345 20934 hybrid_clock.cc:648] HybridClock initialized: now 1786438952098230 us; error 108 us; skew 500 ppm
I20260811 09:02:32.098769 20934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:32.101670 20934 webserver.cc:460] Webserver started at http://127.9.40.66:37745/ using document root <none> and password file <none>
I20260811 09:02:32.102533 20934 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:32.102653 20934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:32.103041 20934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:32.106585 20934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "33f9d2c71f7d4b62b278e8a46fee3d9d"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "fdf3f39cb5351614943c923a80b2be6c"
server_key_iv: "52431a87fbb8e17d825b9d35170870b8"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.107610 20934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "33f9d2c71f7d4b62b278e8a46fee3d9d"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "fdf3f39cb5351614943c923a80b2be6c"
server_key_iv: "52431a87fbb8e17d825b9d35170870b8"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.114336 20934 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:02:32.119706 20951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.121917 20934 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 09:02:32.122228 20934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "33f9d2c71f7d4b62b278e8a46fee3d9d"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "fdf3f39cb5351614943c923a80b2be6c"
server_key_iv: "52431a87fbb8e17d825b9d35170870b8"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.122536 20934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:32.130124 20934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:32.131229 20934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:32.131552 20934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:32.132879 20934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:32.134702 20934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:32.134848 20934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.135001 20934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:32.135092 20934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.175164 20934 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:34347
I20260811 09:02:32.175231 21077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:34347 every 8 connection(s)
I20260811 09:02:32.176847 20934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:02:32.182353 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 20934
I20260811 09:02:32.182896 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:02:32.183629 9377 external_mini_cluster.cc:1428] Setting key d7d9d9b69f1f3c3ebe16b810aa989446
I20260811 09:02:32.187095 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--builtin_ntp_servers=127.9.40.84:34465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:02:32.191110 21080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.181111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:55157 (local address 127.9.40.126:36227)
0811 09:02:32.181890 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.181899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.184198 (+ 2299us) server_negotiation.cc:407] Connection header received
0811 09:02:32.187638 (+ 3440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.187648 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.187744 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.187885 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.190007 (+ 2122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:32.190017 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:32.190025 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:32.190066 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:32.190135 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:32.190155 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:32.190350 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:32.190616 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 09:02:32.190700 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":336,"threads_started":1}
I20260811 09:02:32.192759 21081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.181968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36227 (local address 127.9.40.66:55157)
0811 09:02:32.183955 (+ 1987us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:32.184033 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:32.187111 (+ 3078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:32.188258 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:32.188275 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:32.188456 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:32.189503 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:32.189509 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:32.189583 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:32.189735 (+ 152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:32.190303 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:32.190330 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:32.190340 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:02:32.190753 (+ 413us) client_negotiation.cc:240] Negotiation successful
0811 09:02:32.191151 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1125,"thread_start_us":834,"threads_started":1}
I20260811 09:02:32.194801 21078 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36227
I20260811 09:02:32.195238 21078 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:32.196122 21078 heartbeater.cc:511] Master 127.9.40.126:36227 requested a full tablet report, sending...
I20260811 09:02:32.198328 20716 ts_manager.cc:194] Registered new tserver with Master: 33f9d2c71f7d4b62b278e8a46fee3d9d (127.9.40.66:34347)
W20260811 09:02:32.596019 21082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:32.596392 21082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:32.596572 21082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:32.605046 21082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:32.605260 21082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:32.605470 21082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:02:32.615455 21082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:36227
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:32.617067 21082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:32.619351 21082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:32.635331 21090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:32.635331 21088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:32.635602 21092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:32.636337 21082 server_base.cc:1034] running on GCE node
I20260811 09:02:32.637195 21082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:32.638377 21082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:32.639657 21082 hybrid_clock.cc:648] HybridClock initialized: now 1786438952639590 us; error 63 us; skew 500 ppm
I20260811 09:02:32.640064 21082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:02:32.643257 21082 webserver.cc:460] Webserver started at http://127.9.40.67:42457/ using document root <none> and password file <none>
I20260811 09:02:32.644308 21082 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:32.644481 21082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:32.644938 21082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:32.648460 21082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "f72635abf7ca40cabeee4e28a9568273"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "1d84dcb56005fd0bbee6e1a64849f05e"
server_key_iv: "79cf6002ae4fa9a8b249112c34112660"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.649596 21082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "f72635abf7ca40cabeee4e28a9568273"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "1d84dcb56005fd0bbee6e1a64849f05e"
server_key_iv: "79cf6002ae4fa9a8b249112c34112660"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.656410 21082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260811 09:02:32.661235 21098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.663084 21082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:02:32.663349 21082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "f72635abf7ca40cabeee4e28a9568273"
format_stamp: "Formatted at 2026-08-11 09:02:32 on dist-test-slave-cbsf"
server_key: "1d84dcb56005fd0bbee6e1a64849f05e"
server_key_iv: "79cf6002ae4fa9a8b249112c34112660"
server_key_version: "encryptionkey@0"
I20260811 09:02:32.663676 21082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:32.669064 20928 heartbeater.cc:503] Master 127.9.40.126:36227 was elected leader, sending a full tablet report...
I20260811 09:02:32.671217 21082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:32.672456 21082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:32.672859 21082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:32.674060 21082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:32.675837 21082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:32.675992 21082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.676134 21082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:32.676225 21082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:32.715761 21082 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35407
I20260811 09:02:32.715826 21221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35407 every 8 connection(s)
I20260811 09:02:32.717520 21082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:02:32.718698 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 21082
I20260811 09:02:32.718930 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:02:32.719532 9377 external_mini_cluster.cc:1428] Setting key 37aef69f4a2fd72194cccb8c6263da74
I20260811 09:02:32.729092 21224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.721224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:44735 (local address 127.9.40.126:36227)
0811 09:02:32.722212 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.722217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.723749 (+ 1532us) server_negotiation.cc:407] Connection header received
0811 09:02:32.725732 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.725737 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.725817 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.725983 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.728073 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:32.728080 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:32.728083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:32.728106 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:32.728165 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:32.728179 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:32.728344 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:32.728611 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 09:02:32.728723 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":865,"thread_start_us":749,"threads_started":1}
I20260811 09:02:32.730221 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.722195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:36227 (local address 127.9.40.67:44735)
0811 09:02:32.723552 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:32.723617 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:32.725394 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:32.726318 (+ 924us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:32.726328 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:32.726497 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:32.727525 (+ 1028us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:02:32.727529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:32.727613 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:02:32.727792 (+ 179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:32.728301 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:32.728313 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:32.728322 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:02:32.728677 (+ 355us) client_negotiation.cc:240] Negotiation successful
0811 09:02:32.729056 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":416,"threads_started":1}
I20260811 09:02:32.731928 21222 heartbeater.cc:347] Connected to a master server at 127.9.40.126:36227
I20260811 09:02:32.732357 21222 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:32.733165 21222 heartbeater.cc:511] Master 127.9.40.126:36227 requested a full tablet report, sending...
I20260811 09:02:32.735165 20715 ts_manager.cc:194] Registered new tserver with Master: f72635abf7ca40cabeee4e28a9568273 (127.9.40.67:35407)
I20260811 09:02:32.736605 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:02:32.743960 21224 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:02:32.741988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.9.40.126:36227)
0811 09:02:32.742188 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.742194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.742745 (+ 551us) server_negotiation.cc:407] Connection header received
0811 09:02:32.742976 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.742982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.743076 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.743198 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.743644 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56512: BlockingRecv error: recv got EOF from 127.0.0.1:56512 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:02:32.744521 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20787
I20260811 09:02:32.749732 21233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.746743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.9.40.65:35653)
0811 09:02:32.747461 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.747469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.747502 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:02:32.747582 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.747589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.747826 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.747989 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.749008 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:32.749015 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:32.749028 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:32.749042 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:32.749090 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:32.749105 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:32.749261 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:32.749311 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 09:02:32.749407 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":349,"threads_started":1}
I20260811 09:02:32.751328 20923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:32.838805 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20787
I20260811 09:02:32.863901 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20934
I20260811 09:02:32.868021 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.865015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45900 (local address 127.9.40.66:34347)
0811 09:02:32.865661 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.865669 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.865701 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:02:32.865769 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.865776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.866007 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.866145 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.867222 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:32.867230 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:32.867243 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:32.867255 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:32.867303 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:32.867318 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:32.867437 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:32.867539 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:02:32.867694 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":333,"threads_started":1}
I20260811 09:02:32.869683 21073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:32.957558 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20934
I20260811 09:02:32.982252 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 21082
I20260811 09:02:32.987116 21237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:32.983391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37696 (local address 127.9.40.67:35407)
0811 09:02:32.984267 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:32.984275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:32.984322 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:02:32.984391 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:32.984399 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:32.984652 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:32.984908 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:32.986239 (+ 1331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:32.986246 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:32.986258 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:02:32.986277 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:32.986320 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:02:32.986336 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:32.986549 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:32.986632 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 09:02:32.986749 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":447,"threads_started":1}
I20260811 09:02:32.988763 21217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:33.083436 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 21082
I20260811 09:02:33.106333 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 20681
I20260811 09:02:33.107752 20751 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:33.220201 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 20681
2026-08-11T09:02:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2825 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:02:33 dist-test-slave-cbsf krb5kdc[21242](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:02:33 dist-test-slave-cbsf krb5kdc[21242](info): set up 2 sockets
Aug 11 09:02:33 dist-test-slave-cbsf krb5kdc[21242](info): commencing operation
krb5kdc: starting...
W20260811 09:02:34.478050 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:02:34 dist-test-slave-cbsf krb5kdc[21242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:02:34.816326 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44799
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44799
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:35.226068 21258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:35.226459 21258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:35.226567 21258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:35.234774 21258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:02:35.234954 21258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:35.235014 21258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:35.235085 21258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:02:35.235140 21258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:02:35.246697 21258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44799
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:35.248270 21258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:35.249920 21258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:35.265372 21269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:35.265297 21264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:35.265342 21266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:35.266484 21258 server_base.cc:1034] running on GCE node
I20260811 09:02:35.267560 21258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:35.269559 21258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:35.270819 21258 hybrid_clock.cc:648] HybridClock initialized: now 1786438955270755 us; error 44 us; skew 500 ppm
I20260811 09:02:35.271399 21258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:35 dist-test-slave-cbsf krb5kdc[21242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:35.275384 21258 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:02:35.277437 21258 webserver.cc:460] Webserver started at http://127.9.40.126:35337/ using document root <none> and password file <none>
I20260811 09:02:35.278340 21258 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:35.278482 21258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:35.278951 21258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:35.282675 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "57b229d0965241caa0e11f3c18af9b45"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "afba29f61b269f129f09eb0c5e841290"
server_key_iv: "a2cc9aabb70d526ee8f440d94bdad619"
server_key_version: "encryptionkey@0"
I20260811 09:02:35.283677 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "57b229d0965241caa0e11f3c18af9b45"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "afba29f61b269f129f09eb0c5e841290"
server_key_iv: "a2cc9aabb70d526ee8f440d94bdad619"
server_key_version: "encryptionkey@0"
I20260811 09:02:35.290431 21258 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:02:35.295572 21278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:35.298049 21258 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 09:02:35.298357 21258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "57b229d0965241caa0e11f3c18af9b45"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "afba29f61b269f129f09eb0c5e841290"
server_key_iv: "a2cc9aabb70d526ee8f440d94bdad619"
server_key_version: "encryptionkey@0"
I20260811 09:02:35.298632 21258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:35.330703 21258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:35.335942 21258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:35.336349 21258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:35.358134 21258 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44799
I20260811 09:02:35.358153 21335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44799 every 8 connection(s)
I20260811 09:02:35.359782 21258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:02:35.361726 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 21258
I20260811 09:02:35.362041 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:02:35.362934 9377 external_mini_cluster.cc:1428] Setting key 859003dc310cb538b523c12674ae38ba
I20260811 09:02:35.365650 21336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:02:35 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:35.382040 21336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: Bootstrap starting.
I20260811 09:02:35.386323 21336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:35.387620 21336 log.cc:824] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:35.391237 21340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:35.365509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54424 (local address 127.9.40.126:44799)
0811 09:02:35.366569 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:35.366583 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:35.366637 (+ 54us) server_negotiation.cc:407] Connection header received
0811 09:02:35.368458 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:35.368485 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:35.369110 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:35.369549 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:35.371743 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:35.373543 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:35.375597 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:35.376400 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:35.376578 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:35.381985 (+ 5407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:35.381996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:35.382016 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:35.382071 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:35.385712 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:35.386768 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:35.386777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:35.386786 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:35.387004 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:35.387457 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:35.387464 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:35.387472 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:35.388187 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:35.388562 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:35.389129 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 09:02:35.389683 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":528,"threads_started":1}
I20260811 09:02:35.391765 21336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: No bootstrap required, opened a new log
I20260811 09:02:35.401068 21336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } }
I20260811 09:02:35.401554 21336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:35.401639 21336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57b229d0965241caa0e11f3c18af9b45, State: Initialized, Role: FOLLOWER
I20260811 09:02:35.402307 21336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } }
I20260811 09:02:35.402632 21336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:02:35.402783 21336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:02:35.402993 21336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:35.406008 21336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } }
I20260811 09:02:35.406697 21336 leader_election.cc:302] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57b229d0965241caa0e11f3c18af9b45; no voters:
I20260811 09:02:35.407413 21336 leader_election.cc:288] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:02:35.407601 21342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:35.408453 21342 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [term 1 LEADER]: Becoming Leader. State: Replica: 57b229d0965241caa0e11f3c18af9b45, State: Running, Role: LEADER
I20260811 09:02:35.409124 21342 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } }
I20260811 09:02:35.409988 21336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:02:35.412213 21343 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57b229d0965241caa0e11f3c18af9b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } } }
I20260811 09:02:35.412720 21343 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:35.412422 21344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57b229d0965241caa0e11f3c18af9b45. Latest consensus state: current_term: 1 leader_uuid: "57b229d0965241caa0e11f3c18af9b45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57b229d0965241caa0e11f3c18af9b45" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44799 } } }
I20260811 09:02:35.413254 21344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45 [sys.catalog]: This master's current role is: LEADER
I20260811 09:02:35.415891 21348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:02:35.422838 21348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:02:35.435089 21348 catalog_manager.cc:1347] Generated new cluster ID: c4166f4d428d4367b793272690605da2
I20260811 09:02:35.435283 21348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:02:35.450233 21348 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:02:35.451833 21348 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:02:35.460882 21348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: Generated new TSK 0
I20260811 09:02:35.462271 21348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:02:35.546368 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:35.943065 21366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:35.943404 21366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:35.943527 21366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:35.951450 21366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:35.951617 21366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:35.951741 21366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:02:35.962774 21366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:35.964793 21366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:35.966450 21366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:35.982677 21371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:35.982597 21374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:35.982645 21372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:35.984177 21366 server_base.cc:1034] running on GCE node
I20260811 09:02:35.985074 21366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:35.986280 21366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:35.987563 21366 hybrid_clock.cc:648] HybridClock initialized: now 1786438955987512 us; error 60 us; skew 500 ppm
I20260811 09:02:35.987985 21366 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:35 dist-test-slave-cbsf krb5kdc[21242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:35.992205 21366 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:02:35.994233 21366 webserver.cc:460] Webserver started at http://127.9.40.65:44717/ using document root <none> and password file <none>
I20260811 09:02:35.995132 21366 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:35.995333 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:35.995707 21366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:35.999361 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "8f5dfd34e9254583a2a1fdbddc3db743"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "5d25b901077fffba948890d6b3c3766d"
server_key_iv: "8a8e4e2b0bb55cc9d4ad146234d2428a"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.000388 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "8f5dfd34e9254583a2a1fdbddc3db743"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "5d25b901077fffba948890d6b3c3766d"
server_key_iv: "8a8e4e2b0bb55cc9d4ad146234d2428a"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.007145 21366 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:02:36.011837 21382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.013597 21366 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 09:02:36.013837 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "8f5dfd34e9254583a2a1fdbddc3db743"
format_stamp: "Formatted at 2026-08-11 09:02:35 on dist-test-slave-cbsf"
server_key: "5d25b901077fffba948890d6b3c3766d"
server_key_iv: "8a8e4e2b0bb55cc9d4ad146234d2428a"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.014109 21366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:36.051085 21366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:36.056447 21366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:36.056910 21366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:36.058096 21366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:36.059805 21366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:36.059924 21366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.060104 21366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:36.060182 21366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.099571 21366 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42559
I20260811 09:02:36.099637 21511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42559 every 8 connection(s)
I20260811 09:02:36.101118 21366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:02:36.101934 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 21366
I20260811 09:02:36.102238 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:02:36.103106 9377 external_mini_cluster.cc:1428] Setting key 770f932b2d55d590bea2bafc99e95c47
Aug 11 09:02:36 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438955, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:36.131286 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:36.104423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36385 (local address 127.9.40.126:44799)
0811 09:02:36.105021 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:36.105027 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:36.106484 (+ 1457us) server_negotiation.cc:407] Connection header received
0811 09:02:36.109842 (+ 3358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:36.109847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:36.109956 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:36.110113 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:36.112821 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.113588 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.116583 (+ 2995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.117406 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.117586 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:36.122635 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:36.122647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:36.122656 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:36.122714 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:36.126956 (+ 4242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:36.128117 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:36.128126 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:36.128133 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:36.128325 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:36.128948 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:36.128956 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:36.128962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:36.129411 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:36.129627 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:36.130444 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 09:02:36.130763 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
I20260811 09:02:36.132158 21516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:36.105206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44799 (local address 127.9.40.65:36385)
0811 09:02:36.106251 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:36.106322 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:36.109326 (+ 3004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:36.110500 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:36.110524 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:36.111490 (+ 966us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:36.112528 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:36.112565 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.113801 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.113810 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:36.116264 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:36.116294 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.117766 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.117773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:36.118036 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:36.118951 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:36.118989 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:36.122279 (+ 3290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:36.127237 (+ 4958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:36.127247 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:36.127266 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:36.127914 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:36.128459 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:36.128463 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:36.128489 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:36.128807 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:36.129581 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:36.129588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:36.130001 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 09:02:36.130258 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 09:02:36.130746 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":422,"threads_started":1}
I20260811 09:02:36.134253 21512 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44799
I20260811 09:02:36.134644 21512 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:36.135526 21512 heartbeater.cc:511] Master 127.9.40.126:44799 requested a full tablet report, sending...
I20260811 09:02:36.138403 21298 ts_manager.cc:194] Registered new tserver with Master: 8f5dfd34e9254583a2a1fdbddc3db743 (127.9.40.65:42559)
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:02:36.140820 21298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36385
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:02:36.180603 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:02:36.575167 21519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:36.575619 21519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:36.575729 21519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:36.583779 21519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:36.583962 21519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:36.584159 21519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:02:36.595139 21519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:36.596565 21519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:36.598058 21519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:36.613646 21524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:36.613615 21525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:36.613615 21527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:36.614678 21519 server_base.cc:1034] running on GCE node
I20260811 09:02:36.615422 21519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:36.616590 21519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:36.617841 21519 hybrid_clock.cc:648] HybridClock initialized: now 1786438956617777 us; error 55 us; skew 500 ppm
I20260811 09:02:36.618214 21519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:36 dist-test-slave-cbsf krb5kdc[21242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:36.621990 21519 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:02:36.623776 21519 webserver.cc:460] Webserver started at http://127.9.40.66:46783/ using document root <none> and password file <none>
I20260811 09:02:36.624631 21519 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:36.624843 21519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:36.625226 21519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:36.628667 21519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "a0057dd6ec0d4306a3a601109fa401e7"
format_stamp: "Formatted at 2026-08-11 09:02:36 on dist-test-slave-cbsf"
server_key: "5ccc99827d5bc1ad62f2f087d5307a82"
server_key_iv: "e7212dd79a5738472e1df11bcfeb19ee"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.629746 21519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "a0057dd6ec0d4306a3a601109fa401e7"
format_stamp: "Formatted at 2026-08-11 09:02:36 on dist-test-slave-cbsf"
server_key: "5ccc99827d5bc1ad62f2f087d5307a82"
server_key_iv: "e7212dd79a5738472e1df11bcfeb19ee"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.636014 21519 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260811 09:02:36.640304 21534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.642257 21519 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 09:02:36.642516 21519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "a0057dd6ec0d4306a3a601109fa401e7"
format_stamp: "Formatted at 2026-08-11 09:02:36 on dist-test-slave-cbsf"
server_key: "5ccc99827d5bc1ad62f2f087d5307a82"
server_key_iv: "e7212dd79a5738472e1df11bcfeb19ee"
server_key_version: "encryptionkey@0"
I20260811 09:02:36.642777 21519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:36.655608 21519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:36.660506 21519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:36.660921 21519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:36.662060 21519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:36.663702 21519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:36.663827 21519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.663991 21519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:36.664072 21519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:36.701829 21519 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:38337
I20260811 09:02:36.701874 21664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:38337 every 8 connection(s)
I20260811 09:02:36.703405 21519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:02:36.707062 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 21519
I20260811 09:02:36.707288 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:02:36.707762 9377 external_mini_cluster.cc:1428] Setting key 76e6b3a85771eb8748d8daadff1a50a8
Aug 11 09:02:36 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438956, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:36.728281 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:36.706480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:45363 (local address 127.9.40.126:44799)
0811 09:02:36.707039 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:36.707047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:36.708350 (+ 1303us) server_negotiation.cc:407] Connection header received
0811 09:02:36.710941 (+ 2591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:36.710947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:36.711040 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:36.711183 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:36.713102 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.713888 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.715655 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.716204 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.716361 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:36.720919 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:36.720926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:36.720929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:36.720972 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:36.724510 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:36.725540 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:36.725545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:36.725548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:36.725703 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:36.726252 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:36.726257 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:36.726259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:36.726592 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:36.726778 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:36.727609 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 09:02:36.727889 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":279,"threads_started":1}
I20260811 09:02:36.729543 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:36.707080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44799 (local address 127.9.40.66:45363)
0811 09:02:36.708106 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:36.708167 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:36.710608 (+ 2441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:36.711511 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:36.711526 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:36.712154 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:36.712890 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:36.712920 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.714075 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.714080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:36.715466 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:36.715486 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:36.716484 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:36.716493 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:36.716871 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:36.717765 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:36.717792 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:36.720564 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:36.724763 (+ 4199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:36.724782 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:36.724800 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:36.725357 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:36.725846 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:36.725852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:36.725856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:36.726104 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:36.726747 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:36.726757 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:36.727185 (+ 428us) client_negotiation.cc:769] Sending connection context
0811 09:02:36.727489 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 09:02:36.728149 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":417,"threads_started":1}
I20260811 09:02:36.731495 21665 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44799
I20260811 09:02:36.731887 21665 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:36.732877 21665 heartbeater.cc:511] Master 127.9.40.126:44799 requested a full tablet report, sending...
I20260811 09:02:36.734807 21298 ts_manager.cc:194] Registered new tserver with Master: a0057dd6ec0d4306a3a601109fa401e7 (127.9.40.66:38337)
I20260811 09:02:36.735817 21298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:45363
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:02:36.781028 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:02:37.144678 21512 heartbeater.cc:503] Master 127.9.40.126:44799 was elected leader, sending a full tablet report...
W20260811 09:02:37.165232 21671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:02:37.165575 21671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:02:37.165766 21671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:02:37.173588 21671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:02:37.173779 21671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:02:37.173981 21671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:02:37.185187 21671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44799
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:02:37.186676 21671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:02:37.188181 21671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:02:37.203481 21676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:37.203471 21678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:02:37.203579 21680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:02:37.204854 21671 server_base.cc:1034] running on GCE node
I20260811 09:02:37.205729 21671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:02:37.206859 21671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:02:37.208133 21671 hybrid_clock.cc:648] HybridClock initialized: now 1786438957208082 us; error 57 us; skew 500 ppm
I20260811 09:02:37.208523 21671 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:02:37 dist-test-slave-cbsf krb5kdc[21242](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:02:37.212463 21671 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:02:37.214319 21671 webserver.cc:460] Webserver started at http://127.9.40.67:39687/ using document root <none> and password file <none>
I20260811 09:02:37.215162 21671 fs_manager.cc:359] Metadata directory not provided
I20260811 09:02:37.215333 21671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:02:37.215689 21671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:02:37.219671 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "39fcc7604a81479e9b2348a899a04338"
format_stamp: "Formatted at 2026-08-11 09:02:37 on dist-test-slave-cbsf"
server_key: "ae8872fad3645fa65f44528ac2e0a33f"
server_key_iv: "0a68ef8809febd9baf41e402203dbbba"
server_key_version: "encryptionkey@0"
I20260811 09:02:37.220645 21671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "39fcc7604a81479e9b2348a899a04338"
format_stamp: "Formatted at 2026-08-11 09:02:37 on dist-test-slave-cbsf"
server_key: "ae8872fad3645fa65f44528ac2e0a33f"
server_key_iv: "0a68ef8809febd9baf41e402203dbbba"
server_key_version: "encryptionkey@0"
I20260811 09:02:37.227046 21671 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:02:37.231335 21688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:37.233250 21671 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:02:37.233518 21671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "39fcc7604a81479e9b2348a899a04338"
format_stamp: "Formatted at 2026-08-11 09:02:37 on dist-test-slave-cbsf"
server_key: "ae8872fad3645fa65f44528ac2e0a33f"
server_key_iv: "0a68ef8809febd9baf41e402203dbbba"
server_key_version: "encryptionkey@0"
I20260811 09:02:37.233805 21671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:02:37.279383 21671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:02:37.284858 21671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:02:37.285239 21671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:02:37.286423 21671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:02:37.287986 21671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:02:37.288069 21671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:37.288257 21671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:02:37.288313 21671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:02:37.328361 21671 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38857
I20260811 09:02:37.328428 21813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38857 every 8 connection(s)
I20260811 09:02:37.330008 21671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:02:37.337746 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 21671
I20260811 09:02:37.338032 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:02:37.338582 9377 external_mini_cluster.cc:1428] Setting key 84a258d0f94e758c756e78a0e8ca8915
Aug 11 09:02:37 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438957, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:02:37.355415 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.333211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36621 (local address 127.9.40.126:44799)
0811 09:02:37.333745 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.333750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.334944 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 09:02:37.337148 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.337155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.337266 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.337398 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:37.339686 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.340823 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.342912 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.343351 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.343462 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.347887 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:37.347894 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:37.347898 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:37.347941 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:37.351161 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.352178 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.352182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.352185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.352342 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.353015 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.353019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.353022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.353418 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:37.353591 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.354747 (+ 1156us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.355073 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":296,"threads_started":1}
I20260811 09:02:37.356565 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.333779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44799 (local address 127.9.40.67:36621)
0811 09:02:37.334722 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.334770 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.336741 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.337710 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.337720 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.338538 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:02:37.339418 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.339453 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.341042 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.341050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.342746 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.342763 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.343645 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.343653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.343900 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.344782 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:02:37.344823 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:02:37.347563 (+ 2740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:02:37.351418 (+ 3855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:37.351429 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:37.351449 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:37.352027 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:37.352483 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:02:37.352487 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:02:37.352490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:02:37.352869 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:02:37.353614 (+ 745us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:02:37.353624 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:02:37.354198 (+ 574us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.354592 (+ 394us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.355239 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":423,"threads_started":1}
I20260811 09:02:37.358423 21814 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44799
I20260811 09:02:37.358824 21814 heartbeater.cc:464] Registering TS with master...
I20260811 09:02:37.359753 21814 heartbeater.cc:511] Master 127.9.40.126:44799 requested a full tablet report, sending...
I20260811 09:02:37.361616 21298 ts_manager.cc:194] Registered new tserver with Master: 39fcc7604a81479e9b2348a899a04338 (127.9.40.67:38857)
I20260811 09:02:37.362550 21298 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:36621
I20260811 09:02:37.368080 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:02:37.387140 21816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.373210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54428 (local address 127.9.40.126:44799)
0811 09:02:37.373499 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.373506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.373849 (+ 343us) server_negotiation.cc:407] Connection header received
0811 09:02:37.374012 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.374016 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.374093 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.374186 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:37.375803 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.376596 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.378040 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.378533 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.378667 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.381286 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:37.381293 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:37.381296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:37.381341 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:37.384334 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.385163 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.385167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.385170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.385305 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.385661 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.385665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.385667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.386013 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:37.386219 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.386564 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.386761 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 09:02:37.391959 21298 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:02:37.399770 21298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:02:37.443481 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.414747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38337 (local address 127.0.0.1:54912)
0811 09:02:37.427832 (+ 13085us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.427907 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.428215 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.430794 (+ 2579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.430802 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.430835 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:37.431429 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.431450 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.434351 (+ 2901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.434360 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.438496 (+ 4136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.438539 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.442047 (+ 3508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.442059 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.442317 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.442417 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.442652 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.442907 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12752,"mutex_wait_us":110,"thread_start_us":15173,"threads_started":1}
I20260811 09:02:37.443572 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.414831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54912 (local address 127.9.40.66:38337)
0811 09:02:37.429710 (+ 14879us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.429724 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.429763 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:02:37.429889 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.429898 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.430223 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.430440 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:37.431782 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.434113 (+ 2331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.438899 (+ 4786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.441207 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.441410 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.441683 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.442955 (+ 1272us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.443206 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14691,"thread_start_us":14542,"threads_started":1}
I20260811 09:02:37.444540 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.415583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.9.40.67:38857)
0811 09:02:37.430042 (+ 14459us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.430051 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.430215 (+ 164us) server_negotiation.cc:407] Connection header received
0811 09:02:37.430660 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.430672 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.430958 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.431200 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:37.432495 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.434251 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.438943 (+ 4692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.440817 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.441019 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.441412 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.443954 (+ 2542us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.444176 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14287,"thread_start_us":14090,"threads_started":1}
I20260811 09:02:37.443537 21830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.415240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38857 (local address 127.0.0.1:42280)
0811 09:02:37.430053 (+ 14813us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.430080 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.430235 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.431609 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.431618 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.431664 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:37.432232 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.432250 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.434495 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.434504 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.438558 (+ 4054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.438596 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.441636 (+ 3040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.441648 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.442717 (+ 1069us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.442771 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.442884 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.443060 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14671,"mutex_wait_us":150,"thread_start_us":13257,"threads_started":1}
I20260811 09:02:37.448335 21592 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e (DEFAULT_TABLE table=test-table [id=21d5b082a5cf435182fc492cfa45b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:37.449108 21746 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e (DEFAULT_TABLE table=test-table [id=21d5b082a5cf435182fc492cfa45b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:37.450177 21592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9c50f0f59141d6888d79e7c8066e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:37.450562 21746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9c50f0f59141d6888d79e7c8066e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:37.453224 21835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.430450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42559 (local address 127.0.0.1:60142)
0811 09:02:37.442955 (+ 12505us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.442981 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.443125 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.445120 (+ 1995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.445125 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.445160 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:37.445582 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.445601 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.447721 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.447727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.449956 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.449982 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.452243 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.452252 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.452442 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.452481 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.452603 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.452812 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12361,"thread_start_us":12164,"threads_started":1}
I20260811 09:02:37.453303 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.430819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60142 (local address 127.9.40.65:42559)
0811 09:02:37.442510 (+ 11691us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.442521 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.443286 (+ 765us) server_negotiation.cc:407] Connection header received
0811 09:02:37.443415 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.443424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.443770 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.445140 (+ 1370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:37.445819 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.447495 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.450227 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.451923 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.452111 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.452371 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.452718 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.452921 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11478,"thread_start_us":11262,"threads_started":1}
I20260811 09:02:37.458547 21438 tablet_service.cc:1467] Processing CreateTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e (DEFAULT_TABLE table=test-table [id=21d5b082a5cf435182fc492cfa45b004]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:02:37.460419 21438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e9c50f0f59141d6888d79e7c8066e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:02:37.464380 21837 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Bootstrap starting.
I20260811 09:02:37.466908 21838 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Bootstrap starting.
I20260811 09:02:37.470310 21837 tablet_bootstrap.cc:654] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:37.470999 21838 tablet_bootstrap.cc:654] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:37.471908 21837 log.cc:824] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:37.472280 21838 log.cc:824] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:37.475543 21837 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: No bootstrap required, opened a new log
I20260811 09:02:37.475912 21837 ts_tablet_manager.cc:1397] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260811 09:02:37.475961 21838 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: No bootstrap required, opened a new log
I20260811 09:02:37.476349 21838 ts_tablet_manager.cc:1397] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 09:02:37.477169 21841 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Bootstrap starting.
I20260811 09:02:37.482615 21841 tablet_bootstrap.cc:654] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Neither blocks nor log segments found. Creating new log.
I20260811 09:02:37.484436 21841 log.cc:824] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Log is configured to *not* fsync() on all Append() calls
I20260811 09:02:37.485481 21838 raft_consensus.cc:348] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.486088 21838 raft_consensus.cc:374] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:37.486225 21838 raft_consensus.cc:729] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39fcc7604a81479e9b2348a899a04338, State: Initialized, Role: FOLLOWER
I20260811 09:02:37.487006 21838 consensus_queue.cc:260] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.487210 21837 raft_consensus.cc:348] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.487753 21837 raft_consensus.cc:374] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:37.487874 21837 raft_consensus.cc:729] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0057dd6ec0d4306a3a601109fa401e7, State: Initialized, Role: FOLLOWER
I20260811 09:02:37.488247 21814 heartbeater.cc:503] Master 127.9.40.126:44799 was elected leader, sending a full tablet report...
I20260811 09:02:37.488677 21837 consensus_queue.cc:260] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.489037 21838 ts_tablet_manager.cc:1428] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 09:02:37.489100 21841 tablet_bootstrap.cc:492] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: No bootstrap required, opened a new log
I20260811 09:02:37.489599 21841 ts_tablet_manager.cc:1397] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260811 09:02:37.493243 21665 heartbeater.cc:503] Master 127.9.40.126:44799 was elected leader, sending a full tablet report...
I20260811 09:02:37.494072 21837 ts_tablet_manager.cc:1428] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Time spent starting tablet: real 0.018s user 0.007s sys 0.011s
I20260811 09:02:37.500562 21841 raft_consensus.cc:348] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.501147 21841 raft_consensus.cc:374] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:02:37.501281 21841 raft_consensus.cc:729] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f5dfd34e9254583a2a1fdbddc3db743, State: Initialized, Role: FOLLOWER
I20260811 09:02:37.502046 21841 consensus_queue.cc:260] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.503844 21841 ts_tablet_manager.cc:1428] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260811 09:02:37.582041 21815 tablet.cc:2369] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:02:37.604372 21513 tablet.cc:2369] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:02:37.657169 21843 raft_consensus.cc:482] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:02:37.657594 21843 raft_consensus.cc:504] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.659544 21843 leader_election.cc:288] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0057dd6ec0d4306a3a601109fa401e7 (127.9.40.66:38337), 8f5dfd34e9254583a2a1fdbddc3db743 (127.9.40.65:42559)
I20260811 09:02:37.667057 21817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.659996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38337 (local address 127.9.40.67:32887)
0811 09:02:37.660264 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.660286 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.660472 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.661142 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.661148 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.661184 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:37.661426 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.661439 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.662828 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.662832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.664608 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.664626 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.665994 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.666001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.666232 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.666368 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.666469 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.666683 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":176}
I20260811 09:02:37.667023 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.660791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:40789 (local address 127.9.40.65:42559)
0811 09:02:37.661035 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.661043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.661353 (+ 310us) server_negotiation.cc:407] Connection header received
0811 09:02:37.661532 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.661540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.661627 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.661731 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:37.662199 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.663122 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.665104 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.666027 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.666176 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.666328 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.666611 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.666754 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:02:37.667018 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.659995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:32887 (local address 127.9.40.66:38337)
0811 09:02:37.660267 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.660275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.660358 (+ 83us) server_negotiation.cc:407] Connection header received
0811 09:02:37.660717 (+ 359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.660725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.660832 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.660948 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:02:37.661615 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.662667 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.664841 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.665794 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.665921 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.666000 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.666537 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.666697 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 09:02:37.667169 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.660430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42559 (local address 127.9.40.67:40789)
0811 09:02:37.661236 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 09:02:37.661256 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:02:37.661383 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:02:37.661751 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:02:37.661754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:02:37.661775 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:02:37.662041 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.662048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.663289 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.663295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.664842 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:02:37.664863 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.666244 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.666248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:02:37.666408 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.666432 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:02:37.666556 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 09:02:37.666727 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":6,"thread_start_us":465,"threads_started":1}
I20260811 09:02:37.668638 21615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9c50f0f59141d6888d79e7c8066e4e" candidate_uuid: "39fcc7604a81479e9b2348a899a04338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0057dd6ec0d4306a3a601109fa401e7" is_pre_election: true
I20260811 09:02:37.668653 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9c50f0f59141d6888d79e7c8066e4e" candidate_uuid: "39fcc7604a81479e9b2348a899a04338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" is_pre_election: true
I20260811 09:02:37.669374 21615 raft_consensus.cc:2454] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39fcc7604a81479e9b2348a899a04338 in term 0.
I20260811 09:02:37.669386 21461 raft_consensus.cc:2454] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39fcc7604a81479e9b2348a899a04338 in term 0.
I20260811 09:02:37.670332 21692 leader_election.cc:302] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39fcc7604a81479e9b2348a899a04338, 8f5dfd34e9254583a2a1fdbddc3db743; no voters:
I20260811 09:02:37.670804 21843 raft_consensus.cc:2781] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:02:37.670986 21843 raft_consensus.cc:482] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:02:37.671123 21843 raft_consensus.cc:3037] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:37.673750 21843 raft_consensus.cc:504] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.674542 21843 leader_election.cc:288] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [CANDIDATE]: Term 1 election: Requested vote from peers a0057dd6ec0d4306a3a601109fa401e7 (127.9.40.66:38337), 8f5dfd34e9254583a2a1fdbddc3db743 (127.9.40.65:42559)
I20260811 09:02:37.675261 21461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9c50f0f59141d6888d79e7c8066e4e" candidate_uuid: "39fcc7604a81479e9b2348a899a04338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8f5dfd34e9254583a2a1fdbddc3db743"
I20260811 09:02:37.675259 21615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e9c50f0f59141d6888d79e7c8066e4e" candidate_uuid: "39fcc7604a81479e9b2348a899a04338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0057dd6ec0d4306a3a601109fa401e7"
I20260811 09:02:37.675531 21615 raft_consensus.cc:3037] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:37.675531 21461 raft_consensus.cc:3037] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:02:37.679306 21461 raft_consensus.cc:2454] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39fcc7604a81479e9b2348a899a04338 in term 1.
I20260811 09:02:37.679306 21615 raft_consensus.cc:2454] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39fcc7604a81479e9b2348a899a04338 in term 1.
I20260811 09:02:37.679977 21692 leader_election.cc:302] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39fcc7604a81479e9b2348a899a04338, 8f5dfd34e9254583a2a1fdbddc3db743; no voters:
I20260811 09:02:37.680361 21843 raft_consensus.cc:2781] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:02:37.680945 21843 raft_consensus.cc:686] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 1 LEADER]: Becoming Leader. State: Replica: 39fcc7604a81479e9b2348a899a04338, State: Running, Role: LEADER
I20260811 09:02:37.681506 21843 consensus_queue.cc:237] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } }
I20260811 09:02:37.687112 21298 catalog_manager.cc:5495] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39fcc7604a81479e9b2348a899a04338 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "39fcc7604a81479e9b2348a899a04338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39fcc7604a81479e9b2348a899a04338" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38857 } health_report { overall_health: HEALTHY } } }
W20260811 09:02:37.705924 21666 tablet.cc:2369] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:02:37.729928 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.723977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.9.40.67:38857)
0811 09:02:37.724198 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.724203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.724223 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:02:37.724387 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.724391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.724503 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.724680 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:02:37.725266 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.726032 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.727460 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.727888 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.727991 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.728584 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:02:37.729234 (+ 650us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:02:37.729355 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.729494 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.729594 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:02:37.739856 21615 raft_consensus.cc:1261] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 1 FOLLOWER]: Refusing update from remote peer 39fcc7604a81479e9b2348a899a04338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:37.739900 21461 raft_consensus.cc:1261] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 1 FOLLOWER]: Refusing update from remote peer 39fcc7604a81479e9b2348a899a04338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:02:37.741147 21843 consensus_queue.cc:1035] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0057dd6ec0d4306a3a601109fa401e7" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:37.741597 21847 consensus_queue.cc:1035] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8f5dfd34e9254583a2a1fdbddc3db743" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:02:37.775156 21298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:37.775487 21298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:02:37.779692 21298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57b229d0965241caa0e11f3c18af9b45: Sending DeleteTablet for 3 replicas of tablet 2e9c50f0f59141d6888d79e7c8066e4e
I20260811 09:02:37.781214 21746 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:37 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42280
I20260811 09:02:37.781255 21438 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:37 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:60142
I20260811 09:02:37.781600 21592 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e9c50f0f59141d6888d79e7c8066e4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:02:37 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54912
I20260811 09:02:37.784528 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 21366
I20260811 09:02:37.794117 21863 tablet_replica.cc:330] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: stopping tablet replica
I20260811 09:02:37.794950 21864 tablet_replica.cc:330] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: stopping tablet replica
I20260811 09:02:37.795320 21862 tablet_replica.cc:330] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: stopping tablet replica
I20260811 09:02:37.795732 21864 raft_consensus.cc:2229] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:37.795977 21862 raft_consensus.cc:2229] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:02:37.796288 21863 raft_consensus.cc:2229] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:02:37.796746 21862 raft_consensus.cc:2258] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:37.796892 21864 raft_consensus.cc:2258] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:37.797845 21863 raft_consensus.cc:2258] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:02:37.799665 21862 ts_tablet_manager.cc:1905] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:37.800340 21864 ts_tablet_manager.cc:1905] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:02:37.801806 21863 ts_tablet_manager.cc:1905] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:02:37 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:02:37.807554 21862 ts_tablet_manager.cc:1918] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:37.807927 21862 log.cc:1196] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/2e9c50f0f59141d6888d79e7c8066e4e
I20260811 09:02:37.808655 21862 ts_tablet_manager.cc:1939] T 2e9c50f0f59141d6888d79e7c8066e4e P 8f5dfd34e9254583a2a1fdbddc3db743: Deleting consensus metadata
I20260811 09:02:37.811177 21284 catalog_manager.cc:4841] TS 8f5dfd34e9254583a2a1fdbddc3db743 (127.9.40.65:42559): tablet 2e9c50f0f59141d6888d79e7c8066e4e (table test-table [id=21d5b082a5cf435182fc492cfa45b004]) successfully deleted
I20260811 09:02:37.812844 21864 ts_tablet_manager.cc:1918] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:37.813171 21864 log.cc:1196] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/2e9c50f0f59141d6888d79e7c8066e4e
I20260811 09:02:37.813877 21864 ts_tablet_manager.cc:1939] T 2e9c50f0f59141d6888d79e7c8066e4e P a0057dd6ec0d4306a3a601109fa401e7: Deleting consensus metadata
I20260811 09:02:37.816151 21284 catalog_manager.cc:4841] TS a0057dd6ec0d4306a3a601109fa401e7 (127.9.40.66:38337): tablet 2e9c50f0f59141d6888d79e7c8066e4e (table test-table [id=21d5b082a5cf435182fc492cfa45b004]) successfully deleted
I20260811 09:02:37.816948 21836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.788340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.9.40.65:42559)
0811 09:02:37.788563 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.788571 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.792163 (+ 3592us) server_negotiation.cc:407] Connection header received
0811 09:02:37.792447 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.792454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.792611 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.792790 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:37.795686 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.797072 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.799963 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.800715 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.800977 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.807561 (+ 6584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:37.807585 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:37.807608 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:37.807663 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:37.812214 (+ 4551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.813599 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.813612 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.813616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.813820 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.814351 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.814356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.814360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.814908 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:37.815227 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.816123 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.816497 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:02:37.817679 21863 ts_tablet_manager.cc:1918] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:02:37.817981 21863 log.cc:1196] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/2e9c50f0f59141d6888d79e7c8066e4e
I20260811 09:02:37.818116 21507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:37.818763 21863 ts_tablet_manager.cc:1939] T 2e9c50f0f59141d6888d79e7c8066e4e P 39fcc7604a81479e9b2348a899a04338: Deleting consensus metadata
I20260811 09:02:37.822731 21283 catalog_manager.cc:4841] TS 39fcc7604a81479e9b2348a899a04338 (127.9.40.67:38857): tablet 2e9c50f0f59141d6888d79e7c8066e4e (table test-table [id=21d5b082a5cf435182fc492cfa45b004]) successfully deleted
I20260811 09:02:37.950209 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 21366
I20260811 09:02:37.975826 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 21519
Aug 11 09:02:37 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:02:37.994586 21831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:37.976907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.9.40.66:38337)
0811 09:02:37.977113 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:37.977119 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:37.977168 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:02:37.977245 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:37.977251 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:37.977351 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:37.977469 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:37.978922 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.979772 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.981468 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:37.981907 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:37.982080 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:37.987190 (+ 5110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:37.987213 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:37.987233 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:37.987285 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:37.990839 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.991865 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.991875 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.991878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.992047 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:37.992509 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:37.992513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:37.992515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:37.993194 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:37.993535 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:37.993884 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 09:02:37.994159 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:02:37.995477 21659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:38.090319 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 21519
I20260811 09:02:38.113200 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 21671
Aug 11 09:02:38 dist-test-slave-cbsf krb5kdc[21242](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:02:38.133998 21833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:02:38.114109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.9.40.67:38857)
0811 09:02:38.114328 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:02:38.114334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:02:38.114359 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:02:38.114434 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:02:38.114438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:02:38.114525 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:02:38.114657 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:02:38.116070 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:38.117430 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:38.118844 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:02:38.119633 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:02:38.119810 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:02:38.125231 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:02:38.125259 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:02:38.125291 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:02:38.125374 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:02:38.130170 (+ 4796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:38.131214 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:38.131227 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:38.131231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:38.131461 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:02:38.131903 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:02:38.131908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:02:38.131911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:02:38.132467 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:02:38.132854 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0811 09:02:38.133193 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 09:02:38.133492 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:02:38.135028 21809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:38.266076 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 21671
I20260811 09:02:38.292126 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 21258
I20260811 09:02:38.293550 21330 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:02:38.410409 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 21258
2026-08-11T09:02:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (5190 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:02:38 dist-test-slave-cbsf krb5kdc[21872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:02:38 dist-test-slave-cbsf krb5kdc[21872](info): set up 2 sockets
Aug 11 09:02:38 dist-test-slave-cbsf krb5kdc[21872](info): commencing operation
krb5kdc: starting...
W20260811 09:02:39.639902 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.178s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:02:39 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438959, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:02:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
I20260811 09:02:40.108089 9377 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSMwUEY/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/postgres -l logfile start
2026-08-11 09:02:42.790 UTC [21904] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:02:42.790 UTC [21904] LOG: listening on IPv4 address "127.9.40.84", port 54439
2026-08-11 09:02:42.794 UTC [21904] LOG: listening on Unix socket "/tmp/.s.PGSQL.54439"
2026-08-11 09:02:42.821 UTC [21907] LOG: database system was shut down at 2026-08-11 09:02:41 UTC
2026-08-11 09:02:42.825 UTC [21904] LOG: database system is ready to accept connections
I20260811 09:02:43.921118 9377 mini_postgres.cc:96] Postgres bound to 54439
2026-08-11 09:02:43.949 UTC [21918] FATAL: database "slave" does not exist
127.9.40.84:54439 - accepting connections
I20260811 09:02:43.950129 9377 mini_ranger.cc:161] Starting Ranger...
I20260811 09:02:43.988793 9377 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:02:44.510157 9377 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:02:44.510322 9377 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin
2026-08-11 09:02:44,847 [I] DB FLAVOR :POSTGRES
2026-08-11 09:02:44,847 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:02:44,847 [I] Checking connection..
2026-08-11 09:02:44,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:02:45,145 [I] Checking connection passed.
2026-08-11 09:02:45,146 [I] --------- Verifying version history table ---------
2026-08-11 09:02:45,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:02:45,454 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:02:45,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:02:45,753 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:02:45,754 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:02:45,754 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:02:46.045 UTC [21989] WARNING: there is no transaction in progress
2026-08-11 09:02:46,053 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:02:46,053 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:02:46,352 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:02:46,353 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:02:46,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:02:46,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:02:46,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-11 09:02:47,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:02:47,411 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:02:47,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:02:47,681 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:02:47,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:02:47,963 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:02:48,259 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:02:48,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:02:49.302 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.319 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.335 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.351 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.369 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.387 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.454 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.461 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.477 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.488 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.499 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.513 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.523 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.530 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.536 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:49.543 UTC [22113] WARNING: there is no transaction in progress
2026-08-11 09:02:50,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:02:50,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:02:50,466 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:02:50,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:02:50,743 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:02:50,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:02:51,007 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:02:51,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:02:51,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:02:51,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-11 09:02:51,819 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:02:51,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:02:52,071 [I] DB_PATCHES have already been applied
I20260811 09:02:52.078723 9377 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSMwUEY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:02:52.078846 9377 mini_ranger.cc:193] Using host: 127.9.40.84
I20260811 09:02:52.102221 9377 mini_ranger.cc:227] Ranger admin URL: http://127.9.40.84:58497
Aug 11, 2026 9:02:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:02:52 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:02:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:02:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:02:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:02:53 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:02:53 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:02:53.404011 9377 mini_ranger.cc:160] Time spent starting Ranger: real 9.454s user 0.000s sys 0.076s
Aug 11, 2026 9:03:14 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:03:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 9:03:14 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:03:14 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:03:15 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:03:15.804040 9377 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:03:16 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:03:16.027495 9377 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:03:16.054556 9377 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:03:16.880090 9377 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:03:16.880280 9377 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms
I20260811 09:03:16.880421 9377 mini_ranger_kms.cc:227] Using postgres at 127.9.40.84:54439
2026-08-11 09:03:17,424 [I] DB FLAVOR :POSTGRES
2026-08-11 09:03:17,424 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:03:17,424 [I] Checking connection
2026-08-11 09:03:17,424 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:03:17,669 [I] connection success
2026-08-11 09:03:17,669 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:03:17,669 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:03:17,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:03:17,917 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:03:17,918 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:03:17,918 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:03:17,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:54439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:03:18,205 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:03:18.677354 9377 mini_ranger_kms.cc:310] Created kms service
I20260811 09:03:18.873276 9377 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:03:18.917778 9377 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:03:18 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786438996, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:19.350718 9377 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:03:19.352288 9377 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:03:19.352365 9377 mini_ranger_kms.cc:240] Using host: 127.9.40.84
I20260811 09:03:19.380380 9377 mini_ranger_kms.cc:276] Ranger KMS PID: 22312
I20260811 09:03:19.380596 9377 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.40.84:36275
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:03:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:03:21 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:03:21.839745 9377 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.812s user 0.141s sys 0.446s
I20260811 09:03:21.840085 9377 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:03:21.840322 9377 mini_ranger_kms.cc:401] 127.9.40.84:36275/kms/v1/keys
Aug 11 09:03:22 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:03:22 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439001, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:03:24 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:03:24.859011 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37303
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37303
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:36275/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:03:25.475540 22368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:03:25.475822 22368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:03:25.476016 22368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:03:25.489924 22368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:03:25.490195 22368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:03:25.490358 22368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:03:25.490527 22368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:03:25.490691 22368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:03:25.505539 22368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.40.84:36275/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37303
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:03:25.507098 22368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:03:25.508795 22368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:03:25.526600 22376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:25.526765 22375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:25.526914 22378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:03:25.528265 22368 server_base.cc:1034] running on GCE node
I20260811 09:03:25.529150 22368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:03:25.530651 22368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:03:25.532055 22368 hybrid_clock.cc:648] HybridClock initialized: now 1786439005531902 us; error 31 us; skew 500 ppm
I20260811 09:03:25.532579 22368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:03:25 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439005, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:03:25.537556 22368 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:03:25.539777 22368 webserver.cc:460] Webserver started at http://127.9.40.126:33205/ using document root <none> and password file <none>
I20260811 09:03:25.540613 22368 fs_manager.cc:359] Metadata directory not provided
I20260811 09:03:25.540838 22368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:03:25.541224 22368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:03:25 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439005, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:25.615366 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "258635cd1a294c1da97bb086123e4f3e"
format_stamp: "Formatted at 2026-08-11 09:03:25 on dist-test-slave-cbsf"
server_key: "13fed0d48fca5a56d5a335dd758adc13"
server_key_iv: "60ec50cc4b79a7182120419d358b1656"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:25.616954 22368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "258635cd1a294c1da97bb086123e4f3e"
format_stamp: "Formatted at 2026-08-11 09:03:25 on dist-test-slave-cbsf"
server_key: "13fed0d48fca5a56d5a335dd758adc13"
server_key_iv: "60ec50cc4b79a7182120419d358b1656"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:25.627022 22368 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.002s sys 0.006s
I20260811 09:03:25.653491 22385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:25.655725 22368 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:03:25.656023 22368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "258635cd1a294c1da97bb086123e4f3e"
format_stamp: "Formatted at 2026-08-11 09:03:25 on dist-test-slave-cbsf"
server_key: "13fed0d48fca5a56d5a335dd758adc13"
server_key_iv: "60ec50cc4b79a7182120419d358b1656"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:25.656463 22368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:03:25.676671 22368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:03:25.682754 22368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:03:25.683254 22368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:03:25.737352 22368 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37303
I20260811 09:03:25.737527 22445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37303 every 8 connection(s)
I20260811 09:03:25.739005 22368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:03:25.739593 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 22368
I20260811 09:03:25.739859 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:03:25.740665 9377 external_mini_cluster.cc:1428] Setting key 39d4fafea5e0707cff891ff75fa0f639
I20260811 09:03:25.747449 22446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:03:25 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:03:25.771292 22446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: Bootstrap starting.
I20260811 09:03:25.777300 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:25.743750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45242 (local address 127.9.40.126:37303)
0811 09:03:25.745820 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:25.745838 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:25.745930 (+ 92us) server_negotiation.cc:407] Connection header received
0811 09:03:25.750482 (+ 4552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:25.750515 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:25.751140 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:25.751664 (+ 524us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:25.752997 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:25.756635 (+ 3638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:25.758247 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:25.759955 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:25.760181 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:25.764575 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:25.764593 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:25.764613 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:25.764673 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:25.771430 (+ 6757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:25.772065 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:25.772073 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:25.772081 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:25.772367 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:25.772812 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:25.772817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:25.772820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:25.773950 (+ 1130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:25.774539 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:25.774986 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 09:03:25.775568 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1677,"thread_start_us":1203,"threads_started":1}
I20260811 09:03:25.777782 22446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: Neither blocks nor log segments found. Creating new log.
I20260811 09:03:25.780177 22446 log.cc:824] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:03:25.784662 22446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: No bootstrap required, opened a new log
I20260811 09:03:25.797569 22446 raft_consensus.cc:348] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } }
I20260811 09:03:25.798316 22446 raft_consensus.cc:374] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:03:25.798511 22446 raft_consensus.cc:729] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258635cd1a294c1da97bb086123e4f3e, State: Initialized, Role: FOLLOWER
I20260811 09:03:25.799914 22446 consensus_queue.cc:260] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } }
I20260811 09:03:25.800510 22451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:03:25.801045 22451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } }
I20260811 09:03:25.801986 22451 leader_election.cc:302] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258635cd1a294c1da97bb086123e4f3e; no voters:
I20260811 09:03:25.800420 22446 raft_consensus.cc:388] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:03:25.803398 22446 raft_consensus.cc:482] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:03:25.803840 22451 leader_election.cc:288] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20260811 09:03:25.803804 22446 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:03:25.807694 22446 raft_consensus.cc:504] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } }
I20260811 09:03:25.808168 22446 leader_election.cc:302] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258635cd1a294c1da97bb086123e4f3e; no voters:
I20260811 09:03:25.808251 22452 raft_consensus.cc:2741] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 09:03:25.808746 22446 leader_election.cc:288] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:03:25.809091 22451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:03:25.809556 22451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [term 1 LEADER]: Becoming Leader. State: Replica: 258635cd1a294c1da97bb086123e4f3e, State: Running, Role: LEADER
I20260811 09:03:25.810606 22451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } }
I20260811 09:03:25.812094 22446 sys_catalog.cc:564] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:03:25.814136 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 258635cd1a294c1da97bb086123e4f3e. Latest consensus state: current_term: 1 leader_uuid: "258635cd1a294c1da97bb086123e4f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } } }
I20260811 09:03:25.815058 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [sys.catalog]: This master's current role is: LEADER
I20260811 09:03:25.814016 22452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "258635cd1a294c1da97bb086123e4f3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258635cd1a294c1da97bb086123e4f3e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37303 } } }
I20260811 09:03:25.816581 22452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e [sys.catalog]: This master's current role is: LEADER
I20260811 09:03:25.822631 22457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:03:25.834326 22457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:03:25.835613 22446 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:03:25.866905 22457 catalog_manager.cc:1347] Generated new cluster ID: 6415ce3c16714298a4ab16b509df6679
I20260811 09:03:25.867156 22457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:03:25.886940 22457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:03:25.888345 22457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:03:25.907997 22457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: Generated new TSK 0
I20260811 09:03:25.909523 22457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:03:26.782552 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:36275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:03:27 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:03:27.516439 22497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:03:27.516958 22497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:03:27.517261 22497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:03:27.528275 22497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:03:27.528499 22497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:03:27.528779 22497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:03:27.542276 22497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.40.84:36275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:03:27.543839 22497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:03:27.545253 22497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:03:27.579708 22509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:27.584548 22508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:03:27.584980 22497 server_base.cc:1034] running on GCE node
W20260811 09:03:27.586032 22511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:03:27.586736 22497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:03:27.593396 22497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:03:27.594761 22497 hybrid_clock.cc:648] HybridClock initialized: now 1786439007594695 us; error 110 us; skew 500 ppm
I20260811 09:03:27.595188 22497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:03:27 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:03:27.601413 22497 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:03:27.607124 22497 webserver.cc:460] Webserver started at http://127.9.40.65:40761/ using document root <none> and password file <none>
I20260811 09:03:27.608253 22497 fs_manager.cc:359] Metadata directory not provided
I20260811 09:03:27.608577 22497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:03:27.609197 22497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:03:27 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:27.635491 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e"
format_stamp: "Formatted at 2026-08-11 09:03:27 on dist-test-slave-cbsf"
server_key: "af3eb8b5f683c445a9b7009a674d7d2c"
server_key_iv: "5a3dd2b6f2949c093415c9c8e0b2b18e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:27.638563 22497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e"
format_stamp: "Formatted at 2026-08-11 09:03:27 on dist-test-slave-cbsf"
server_key: "af3eb8b5f683c445a9b7009a674d7d2c"
server_key_iv: "5a3dd2b6f2949c093415c9c8e0b2b18e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:27.659441 22497 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.007s sys 0.011s
I20260811 09:03:27.694605 22520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:27.698262 22497 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.002s sys 0.007s
I20260811 09:03:27.698719 22497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e"
format_stamp: "Formatted at 2026-08-11 09:03:27 on dist-test-slave-cbsf"
server_key: "af3eb8b5f683c445a9b7009a674d7d2c"
server_key_iv: "5a3dd2b6f2949c093415c9c8e0b2b18e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:27.699329 22497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:03:27.755883 22497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:03:27.763005 22497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:03:27.763525 22497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:03:27.771852 22497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:03:27.781605 22497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:03:27.781891 22497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:03:27.782295 22497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:03:27.782531 22497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:03:27 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:03:28 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:28.533859 22497 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38795
I20260811 09:03:28.534101 22650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38795 every 8 connection(s)
I20260811 09:03:28.536387 22497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:03:28.538703 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 22497
I20260811 09:03:28.538998 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:03:28.540397 9377 external_mini_cluster.cc:1428] Setting key 8514929fdca9ee6f839d2ab04d675706
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Aug 11 09:03:28 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439007, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:03:28.608384 22654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:28.541849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37303 (local address 127.9.40.65:56885)
0811 09:03:28.554155 (+ 12306us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:28.554209 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:28.566603 (+ 12394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:28.567931 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:28.567946 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:28.568721 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:03:28.569679 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:28.569710 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:28.571466 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:28.571474 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:28.573824 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:28.573855 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:28.575313 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:28.575322 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:28.575825 (+ 503us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:28.585665 (+ 9840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:03:28.585702 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:03:28.594198 (+ 8496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:03:28.601415 (+ 7217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:28.601429 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:28.601451 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:28.602355 (+ 904us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:28.603011 (+ 656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:28.603017 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:28.603021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:28.603572 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:28.604494 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:03:28.604504 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:03:28.605379 (+ 875us) client_negotiation.cc:769] Sending connection context
0811 09:03:28.605687 (+ 308us) client_negotiation.cc:240] Negotiation successful
0811 09:03:28.606324 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11923,"thread_start_us":11596,"threads_started":1}
I20260811 09:03:28.610072 22653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:28.540825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56885 (local address 127.9.40.126:37303)
0811 09:03:28.564203 (+ 23378us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:28.564213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:28.564242 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:03:28.567087 (+ 2845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:28.567096 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:28.567200 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:28.567431 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:28.569895 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:28.571231 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:28.574009 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:28.574797 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:28.574918 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:28.594675 (+ 19757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:28.594685 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:28.594692 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:28.594743 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:28.601127 (+ 6384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:28.602551 (+ 1424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:28.602557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:28.602579 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:28.602849 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:28.603760 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:28.603764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:28.603766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:28.604328 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:28.609053 (+ 4725us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:28.609219 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:03:28.609492 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23144,"thread_start_us":24102,"threads_started":1}
I20260811 09:03:28.611835 22651 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37303
I20260811 09:03:28.612277 22651 heartbeater.cc:464] Registering TS with master...
I20260811 09:03:28.613868 22651 heartbeater.cc:511] Master 127.9.40.126:37303 requested a full tablet report, sending...
I20260811 09:03:28.616544 22406 ts_manager.cc:194] Registered new tserver with Master: 4d8d6ffdd1d643518f0b2aa5d36c032e (127.9.40.65:38795)
I20260811 09:03:28.619169 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:56885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:03:28.635869 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:36275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:03:29.324833 22665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:03:29.325209 22665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:03:29.325484 22665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:03:29.335707 22665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:03:29.335947 22665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:03:29.336182 22665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:03:29.349092 22665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.40.84:36275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:03:29.350517 22665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:03:29.351924 22665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:03:29.371974 22672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:29.372104 22673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:29.373096 22675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:03:29.373556 22665 server_base.cc:1034] running on GCE node
I20260811 09:03:29.374218 22665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:03:29.375178 22665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:03:29.376437 22665 hybrid_clock.cc:648] HybridClock initialized: now 1786439009376339 us; error 34 us; skew 500 ppm
I20260811 09:03:29.377013 22665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:03:29 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:03:29.381521 22665 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:03:29.383400 22665 webserver.cc:460] Webserver started at http://127.9.40.66:46173/ using document root <none> and password file <none>
I20260811 09:03:29.384311 22665 fs_manager.cc:359] Metadata directory not provided
I20260811 09:03:29.384543 22665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:03:29.385030 22665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:03:29 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:29.404102 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "0fea02af4163446092de2c7e3e72514a"
format_stamp: "Formatted at 2026-08-11 09:03:29 on dist-test-slave-cbsf"
server_key: "923760a5702a568b4ddfaef6a07d131e"
server_key_iv: "232bb63c0497c4d960d8970840313e3e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:29.405475 22665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "0fea02af4163446092de2c7e3e72514a"
format_stamp: "Formatted at 2026-08-11 09:03:29 on dist-test-slave-cbsf"
server_key: "923760a5702a568b4ddfaef6a07d131e"
server_key_iv: "232bb63c0497c4d960d8970840313e3e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:29.414585 22665 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.001s sys 0.006s
I20260811 09:03:29.434628 22683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:29.436297 22665 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 09:03:29.436527 22665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "0fea02af4163446092de2c7e3e72514a"
format_stamp: "Formatted at 2026-08-11 09:03:29 on dist-test-slave-cbsf"
server_key: "923760a5702a568b4ddfaef6a07d131e"
server_key_iv: "232bb63c0497c4d960d8970840313e3e"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:29.436959 22665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:03:29.465853 22665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:03:29.471257 22665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:03:29.471704 22665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:03:29.473256 22665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:03:29.475685 22665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:03:29.475821 22665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:29.476066 22665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:03:29.476186 22665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:29.564486 22665 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:42705
I20260811 09:03:29.564677 22811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:42705 every 8 connection(s)
I20260811 09:03:29.566083 22665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:03:29.576021 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 22665
I20260811 09:03:29.576223 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:03:29.576849 9377 external_mini_cluster.cc:1428] Setting key b81d4a8f5a007ca167f584dc8a573934
Aug 11 09:03:29 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439009, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:03:29.597059 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:29.570484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:38369 (local address 127.9.40.126:37303)
0811 09:03:29.571508 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:29.571517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:29.572643 (+ 1126us) server_negotiation.cc:407] Connection header received
0811 09:03:29.575477 (+ 2834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:29.575484 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:29.575599 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:29.575723 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:29.578144 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:29.579422 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:29.581694 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:29.582689 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:29.582879 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:29.587630 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:29.587640 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:29.587647 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:29.587723 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:29.593184 (+ 5461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:29.593993 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:29.593999 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:29.594009 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:29.594241 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:29.594783 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:29.594787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:29.594789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:29.595161 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:29.595326 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:29.596174 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 09:03:29.596499 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":577,"threads_started":1}
I20260811 09:03:29.598318 22817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:29.570901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37303 (local address 127.9.40.66:38369)
0811 09:03:29.572465 (+ 1564us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:29.572504 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:29.575196 (+ 2692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:29.575959 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:29.575967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:29.576837 (+ 870us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:03:29.577947 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:29.577969 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:29.579689 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:29.579695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:29.581541 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:29.581554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:29.583050 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:29.583057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:29.583392 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:29.584280 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:03:29.584304 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:03:29.587338 (+ 3034us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:03:29.593342 (+ 6004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:29.593350 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:29.593372 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:29.593858 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:29.594368 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:29.594373 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:29.594377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:29.594647 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:29.595308 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:03:29.595316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:03:29.595683 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 09:03:29.595960 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 09:03:29.596645 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"thread_start_us":841,"threads_started":1}
I20260811 09:03:29.600296 22812 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37303
I20260811 09:03:29.600669 22812 heartbeater.cc:464] Registering TS with master...
I20260811 09:03:29.601732 22812 heartbeater.cc:511] Master 127.9.40.126:37303 requested a full tablet report, sending...
I20260811 09:03:29.603400 22406 ts_manager.cc:194] Registered new tserver with Master: 0fea02af4163446092de2c7e3e72514a (127.9.40.66:42705)
I20260811 09:03:29.604434 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:38369
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
I20260811 09:03:29.623909 22651 heartbeater.cc:503] Master 127.9.40.126:37303 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:03:29.656065 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:36275/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39247
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:03:30.313591 22820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:03:30.313992 22820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:03:30.314235 22820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:03:30.324280 22820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:03:30.324530 22820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:03:30.324854 22820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:03:30.337677 22820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39247
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.40.84:36275/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:03:30.339160 22820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:03:30.340585 22820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:03:30.360721 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:30.360728 22826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:03:30.365029 22828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:03:30.366600 22820 server_base.cc:1034] running on GCE node
I20260811 09:03:30.367486 22820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:03:30.368626 22820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:03:30.369931 22820 hybrid_clock.cc:648] HybridClock initialized: now 1786439010369783 us; error 77 us; skew 500 ppm
I20260811 09:03:30.370399 22820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:03:30 dist-test-slave-cbsf krb5kdc[21872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:03:30.375016 22820 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:03:30.376932 22820 webserver.cc:460] Webserver started at http://127.9.40.67:46353/ using document root <none> and password file <none>
I20260811 09:03:30.377827 22820 fs_manager.cc:359] Metadata directory not provided
I20260811 09:03:30.378090 22820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:03:30.378525 22820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:03:30 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:03:30.398200 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "fb8191fd6d324abca7f74d0782bf6f72"
format_stamp: "Formatted at 2026-08-11 09:03:30 on dist-test-slave-cbsf"
server_key: "a11e6eb0753f74b97c7bca851871146a"
server_key_iv: "cd041335b47a77500a562b1386155cbe"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:30.399542 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "fb8191fd6d324abca7f74d0782bf6f72"
format_stamp: "Formatted at 2026-08-11 09:03:30 on dist-test-slave-cbsf"
server_key: "a11e6eb0753f74b97c7bca851871146a"
server_key_iv: "cd041335b47a77500a562b1386155cbe"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:30.408846 22820 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.006s sys 0.005s
I20260811 09:03:30.428061 22835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:30.429853 22820 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:03:30.430097 22820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "fb8191fd6d324abca7f74d0782bf6f72"
format_stamp: "Formatted at 2026-08-11 09:03:30 on dist-test-slave-cbsf"
server_key: "a11e6eb0753f74b97c7bca851871146a"
server_key_iv: "cd041335b47a77500a562b1386155cbe"
server_key_version: "kuduclusterkey@0"
I20260811 09:03:30.430408 22820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:03:30.457406 22820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:03:30.462818 22820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:03:30.463297 22820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:03:30.464974 22820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:03:30.466836 22820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:03:30.466928 22820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:30.467121 22820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:03:30.467227 22820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:03:30.570233 22820 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:39671
I20260811 09:03:30.570392 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:39671 every 8 connection(s)
I20260811 09:03:30.572467 22820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:03:30.573627 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 22820
I20260811 09:03:30.573891 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:03:30.574846 9377 external_mini_cluster.cc:1428] Setting key 8b34449a5f155e935651e0af325b3e40
Aug 11 09:03:30 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439010, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:03:30.601378 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.576611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46599 (local address 127.9.40.126:37303)
0811 09:03:30.577653 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.577660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.579571 (+ 1911us) server_negotiation.cc:407] Connection header received
0811 09:03:30.582362 (+ 2791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.582366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.582501 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.582686 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:30.584508 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.585703 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.587908 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.588445 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.588545 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.593802 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:30.593807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:30.593811 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:30.593866 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:30.597798 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:30.598721 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:30.598726 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:30.598728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:30.598888 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:30.599406 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:30.599409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:30.599411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:30.599839 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:30.599986 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.600655 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.600947 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":706,"threads_started":1}
I20260811 09:03:30.602564 22962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.577777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37303 (local address 127.9.40.67:46599)
0811 09:03:30.579345 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.579381 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.582115 (+ 2734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.582882 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.582889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.583514 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:03:30.584346 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.584364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.585864 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.585868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.587727 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.587739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.588752 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.588759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.589062 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.589932 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:03:30.589955 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:03:30.593469 (+ 3514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:03:30.597979 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:30.597987 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:30.598000 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:30.598614 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:30.598996 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:03:30.598999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:03:30.599001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:03:30.599288 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:03:30.599962 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:03:30.599969 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:03:30.600309 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.600566 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.601060 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1295,"thread_start_us":926,"threads_started":1}
I20260811 09:03:30.604341 22958 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37303
I20260811 09:03:30.604741 22958 heartbeater.cc:464] Registering TS with master...
I20260811 09:03:30.605676 22958 heartbeater.cc:511] Master 127.9.40.126:37303 requested a full tablet report, sending...
I20260811 09:03:30.607295 22406 ts_manager.cc:194] Registered new tserver with Master: fb8191fd6d324abca7f74d0782bf6f72 (127.9.40.67:39671)
I20260811 09:03:30.607527 22812 heartbeater.cc:503] Master 127.9.40.126:37303 was elected leader, sending a full tablet report...
I20260811 09:03:30.608680 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:46599
I20260811 09:03:30.616962 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:03:30.640205 22960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.623093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36020 (local address 127.9.40.126:37303)
0811 09:03:30.623262 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.623267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.624363 (+ 1096us) server_negotiation.cc:407] Connection header received
0811 09:03:30.624615 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.624620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.624758 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.624871 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:30.627263 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.628977 (+ 1714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.630777 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.631220 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.631315 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.634158 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:30.634163 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:30.634166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:30.634197 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:30.637544 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:30.638289 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:30.638293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:30.638296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:30.638475 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:30.638941 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:30.638944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:30.638946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:30.639235 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:30.639427 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.639755 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.639955 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:03:30.645291 22404 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:03:30.656164 22404 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:03:30.695662 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.674673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54548 (local address 127.9.40.67:39671)
0811 09:03:30.682421 (+ 7748us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.682432 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.682490 (+ 58us) server_negotiation.cc:407] Connection header received
0811 09:03:30.682649 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.682660 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.682983 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.684281 (+ 1298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:03:30.685000 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.687008 (+ 2008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.691572 (+ 4564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.693888 (+ 2316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.694038 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.694402 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.694825 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.695034 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7485,"thread_start_us":7064,"threads_started":1}
I20260811 09:03:30.695839 22974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.674205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:39671 (local address 127.0.0.1:54548)
0811 09:03:30.678358 (+ 4153us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.678375 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.678609 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.683433 (+ 4824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.683442 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.683480 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:03:30.684413 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.684428 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.687174 (+ 2746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.687179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.691358 (+ 4179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.691372 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.694220 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.694228 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.694485 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.694570 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.694737 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.694957 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3977,"mutex_wait_us":48,"thread_start_us":3688,"threads_started":1}
I20260811 09:03:30.696552 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.675605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46164 (local address 127.9.40.66:42705)
0811 09:03:30.679675 (+ 4070us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.679689 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.679725 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:03:30.683340 (+ 3615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.683358 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.683629 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.684020 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:03:30.685650 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.687911 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.692184 (+ 4273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.694040 (+ 1856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.694230 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.694560 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.695635 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.695874 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3820,"thread_start_us":4080,"threads_started":1}
I20260811 09:03:30.696938 22973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.674058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42705 (local address 127.0.0.1:46164)
0811 09:03:30.675576 (+ 1518us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.675649 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.676026 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.684038 (+ 8012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.684051 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.684086 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:03:30.685121 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.685156 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.688055 (+ 2899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.688064 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.691895 (+ 3831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.691908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.694682 (+ 2774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.694690 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.695034 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.695080 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.695167 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.695334 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1181,"mutex_wait_us":69,"thread_start_us":959,"threads_started":1}
I20260811 09:03:30.701202 22739 tablet_service.cc:1467] Processing CreateTablet for tablet dab6cb559387496da9a43a5526c6ea07 (DEFAULT_TABLE table=test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:03:30.701333 22888 tablet_service.cc:1467] Processing CreateTablet for tablet dab6cb559387496da9a43a5526c6ea07 (DEFAULT_TABLE table=test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:03:30.702448 22977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.678436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38795 (local address 127.0.0.1:42276)
0811 09:03:30.683712 (+ 5276us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.683732 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.684019 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.686164 (+ 2145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.686169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.686194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:03:30.687208 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.687222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.691127 (+ 3905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.691134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.695004 (+ 3870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.695018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.701077 (+ 6059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.701086 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.701407 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.701442 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.701562 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.701714 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5052,"mutex_wait_us":3,"thread_start_us":4801,"threads_started":1}
I20260811 09:03:30.703024 22739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab6cb559387496da9a43a5526c6ea07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:03:30.703614 22888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab6cb559387496da9a43a5526c6ea07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:03:30.703857 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.679230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.9.40.65:38795)
0811 09:03:30.683143 (+ 3913us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.683160 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.684727 (+ 1567us) server_negotiation.cc:407] Connection header received
0811 09:03:30.684869 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.684879 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.685253 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.685624 (+ 371us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:03:30.687524 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.690836 (+ 3312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.695192 (+ 4356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.697684 (+ 2492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.697856 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.702882 (+ 5026us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.703015 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.703267 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3660,"thread_start_us":3390,"threads_started":1}
I20260811 09:03:30.708820 22572 tablet_service.cc:1467] Processing CreateTablet for tablet dab6cb559387496da9a43a5526c6ea07 (DEFAULT_TABLE table=test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:03:30.710839 22572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dab6cb559387496da9a43a5526c6ea07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:03:30.722092 22979 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Bootstrap starting.
I20260811 09:03:30.725301 22980 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Bootstrap starting.
I20260811 09:03:30.728930 22979 tablet_bootstrap.cc:654] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Neither blocks nor log segments found. Creating new log.
I20260811 09:03:30.731348 22979 log.cc:824] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Log is configured to *not* fsync() on all Append() calls
I20260811 09:03:30.732339 22980 tablet_bootstrap.cc:654] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Neither blocks nor log segments found. Creating new log.
I20260811 09:03:30.735637 22980 log.cc:824] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:03:30.742961 22979 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: No bootstrap required, opened a new log
I20260811 09:03:30.742974 22980 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: No bootstrap required, opened a new log
I20260811 09:03:30.743301 22980 ts_tablet_manager.cc:1397] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.011s
I20260811 09:03:30.743327 22979 ts_tablet_manager.cc:1397] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.006s
I20260811 09:03:30.757802 22982 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Bootstrap starting.
I20260811 09:03:30.759764 22980 raft_consensus.cc:348] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.760484 22980 raft_consensus.cc:374] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:03:30.760205 22979 raft_consensus.cc:348] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.760797 22980 raft_consensus.cc:729] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fea02af4163446092de2c7e3e72514a, State: Initialized, Role: FOLLOWER
I20260811 09:03:30.760912 22979 raft_consensus.cc:374] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:03:30.761196 22979 raft_consensus.cc:729] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8191fd6d324abca7f74d0782bf6f72, State: Initialized, Role: FOLLOWER
I20260811 09:03:30.761847 22980 consensus_queue.cc:260] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.762111 22979 consensus_queue.cc:260] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.764945 22982 tablet_bootstrap.cc:654] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Neither blocks nor log segments found. Creating new log.
I20260811 09:03:30.767477 22982 log.cc:824] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:03:30.768800 22979 ts_tablet_manager.cc:1428] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Time spent starting tablet: real 0.025s user 0.012s sys 0.014s
I20260811 09:03:30.769512 22958 heartbeater.cc:503] Master 127.9.40.126:37303 was elected leader, sending a full tablet report...
I20260811 09:03:30.780556 22980 ts_tablet_manager.cc:1428] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Time spent starting tablet: real 0.037s user 0.016s sys 0.021s
I20260811 09:03:30.785056 22982 tablet_bootstrap.cc:492] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: No bootstrap required, opened a new log
I20260811 09:03:30.785466 22982 ts_tablet_manager.cc:1397] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.011s
I20260811 09:03:30.796118 22982 raft_consensus.cc:348] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.796768 22982 raft_consensus.cc:374] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:03:30.797092 22982 raft_consensus.cc:729] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8d6ffdd1d643518f0b2aa5d36c032e, State: Initialized, Role: FOLLOWER
I20260811 09:03:30.798120 22982 consensus_queue.cc:260] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.800580 22982 ts_tablet_manager.cc:1428] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
W20260811 09:03:30.817842 22652 tablet.cc:2369] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:03:30.820149 22813 tablet.cc:2369] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:03:30.825881 22959 tablet.cc:2369] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:03:30.916018 22987 raft_consensus.cc:482] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:03:30.916602 22987 raft_consensus.cc:504] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.919402 22987 leader_election.cc:288] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb8191fd6d324abca7f74d0782bf6f72 (127.9.40.67:39671), 0fea02af4163446092de2c7e3e72514a (127.9.40.66:42705)
I20260811 09:03:30.930372 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.920007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39343 (local address 127.9.40.67:39671)
0811 09:03:30.920391 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.920397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.921247 (+ 850us) server_negotiation.cc:407] Connection header received
0811 09:03:30.921580 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.921587 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.921664 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.921712 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:03:30.922850 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.924492 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.927441 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.929409 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.929537 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.929631 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.930000 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.930113 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260811 09:03:30.930613 22988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.919601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:39671 (local address 127.9.40.65:39343)
0811 09:03:30.921107 (+ 1506us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.921124 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.921354 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.921967 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.921970 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.921998 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:03:30.922677 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.922708 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.924770 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.924776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.927269 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.927281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.929645 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.929651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.929834 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.929866 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.929960 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.930101 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1344,"mutex_wait_us":3,"thread_start_us":1678,"threads_started":1}
I20260811 09:03:30.932102 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.920013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:48655 (local address 127.9.40.66:42705)
0811 09:03:30.920357 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.920365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.922594 (+ 2229us) server_negotiation.cc:407] Connection header received
0811 09:03:30.922779 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.922787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.922871 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.922944 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:03:30.924341 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.925793 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.929533 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.930952 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.931162 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.931271 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.931567 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.931702 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:03:30.931975 22910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab6cb559387496da9a43a5526c6ea07" candidate_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8191fd6d324abca7f74d0782bf6f72" is_pre_election: true
I20260811 09:03:30.932487 22989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.920604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42705 (local address 127.9.40.65:48655)
0811 09:03:30.922437 (+ 1833us) negotiation.cc:107] Waiting for socket to connect
0811 09:03:30.922453 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:03:30.922638 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:03:30.923027 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:03:30.923032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:03:30.923067 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:03:30.924108 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.924123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.926000 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.926005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.929341 (+ 3336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:03:30.929359 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.931297 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.931302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:03:30.931415 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.931437 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:03:30.931543 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 09:03:30.931662 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1666,"mutex_wait_us":58,"thread_start_us":1233,"threads_started":1}
I20260811 09:03:30.932766 22910 raft_consensus.cc:2454] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8d6ffdd1d643518f0b2aa5d36c032e in term 0.
I20260811 09:03:30.933837 22525 leader_election.cc:302] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8d6ffdd1d643518f0b2aa5d36c032e, fb8191fd6d324abca7f74d0782bf6f72; no voters:
I20260811 09:03:30.934124 22760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab6cb559387496da9a43a5526c6ea07" candidate_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fea02af4163446092de2c7e3e72514a" is_pre_election: true
I20260811 09:03:30.934628 22987 raft_consensus.cc:2781] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:03:30.934933 22987 raft_consensus.cc:482] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:03:30.935168 22987 raft_consensus.cc:3037] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:03:30.935118 22760 raft_consensus.cc:2454] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d8d6ffdd1d643518f0b2aa5d36c032e in term 0.
I20260811 09:03:30.938804 22987 raft_consensus.cc:504] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.939635 22987 leader_election.cc:288] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [CANDIDATE]: Term 1 election: Requested vote from peers fb8191fd6d324abca7f74d0782bf6f72 (127.9.40.67:39671), 0fea02af4163446092de2c7e3e72514a (127.9.40.66:42705)
I20260811 09:03:30.940234 22910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab6cb559387496da9a43a5526c6ea07" candidate_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8191fd6d324abca7f74d0782bf6f72"
I20260811 09:03:30.940237 22760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dab6cb559387496da9a43a5526c6ea07" candidate_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fea02af4163446092de2c7e3e72514a"
I20260811 09:03:30.940479 22910 raft_consensus.cc:3037] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:03:30.940511 22760 raft_consensus.cc:3037] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:03:30.944288 22910 raft_consensus.cc:2454] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8d6ffdd1d643518f0b2aa5d36c032e in term 1.
I20260811 09:03:30.944720 22760 raft_consensus.cc:2454] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d8d6ffdd1d643518f0b2aa5d36c032e in term 1.
I20260811 09:03:30.944823 22525 leader_election.cc:302] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d8d6ffdd1d643518f0b2aa5d36c032e, fb8191fd6d324abca7f74d0782bf6f72; no voters:
I20260811 09:03:30.945271 22987 raft_consensus.cc:2781] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:03:30.946353 22987 raft_consensus.cc:686] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 1 LEADER]: Becoming Leader. State: Replica: 4d8d6ffdd1d643518f0b2aa5d36c032e, State: Running, Role: LEADER
I20260811 09:03:30.947019 22987 consensus_queue.cc:237] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } }
I20260811 09:03:30.953330 22404 catalog_manager.cc:5495] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d8d6ffdd1d643518f0b2aa5d36c032e (127.9.40.65). New cstate: current_term: 1 leader_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4d8d6ffdd1d643518f0b2aa5d36c032e" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38795 } health_report { overall_health: HEALTHY } } }
I20260811 09:03:30.989041 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:30.982150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.9.40.65:38795)
0811 09:03:30.982440 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:30.982453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:30.982468 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:03:30.982551 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:30.982554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:30.982662 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:30.982845 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:03:30.983401 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.984468 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.986455 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:30.987113 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:30.987254 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:30.987783 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:03:30.988337 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:03:30.988512 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:30.988621 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 09:03:30.988724 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:03:31.002218 22910 raft_consensus.cc:1261] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 1 FOLLOWER]: Refusing update from remote peer 4d8d6ffdd1d643518f0b2aa5d36c032e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:03:31.002646 22760 raft_consensus.cc:1261] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 1 FOLLOWER]: Refusing update from remote peer 4d8d6ffdd1d643518f0b2aa5d36c032e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:03:31.003830 22991 consensus_queue.cc:1035] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb8191fd6d324abca7f74d0782bf6f72" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:03:31.004552 22987 consensus_queue.cc:1035] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fea02af4163446092de2c7e3e72514a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:03:31.042219 22406 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:03:31.042593 22406 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:03:31.047164 22406 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 258635cd1a294c1da97bb086123e4f3e: Sending DeleteTablet for 3 replicas of tablet dab6cb559387496da9a43a5526c6ea07
I20260811 09:03:31.048607 22888 tablet_service.cc:1514] Processing DeleteTablet for tablet dab6cb559387496da9a43a5526c6ea07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:03:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54548
I20260811 09:03:31.048908 22739 tablet_service.cc:1514] Processing DeleteTablet for tablet dab6cb559387496da9a43a5526c6ea07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:03:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:46164
I20260811 09:03:31.050191 22572 tablet_service.cc:1514] Processing DeleteTablet for tablet dab6cb559387496da9a43a5526c6ea07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:03:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42276
I20260811 09:03:31.050654 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 22497
I20260811 09:03:31.057019 23007 tablet_replica.cc:330] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: stopping tablet replica
I20260811 09:03:31.057494 23007 raft_consensus.cc:2229] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:03:31.058320 23007 raft_consensus.cc:2258] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:03:31.060714 23007 ts_tablet_manager.cc:1905] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:03:31.069800 23007 ts_tablet_manager.cc:1918] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:03:31.070030 23007 log.cc:1196] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/dab6cb559387496da9a43a5526c6ea07
I20260811 09:03:31.070662 23007 ts_tablet_manager.cc:1939] T dab6cb559387496da9a43a5526c6ea07 P 4d8d6ffdd1d643518f0b2aa5d36c032e: Deleting consensus metadata
I20260811 09:03:31.071287 23004 tablet_replica.cc:330] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: stopping tablet replica
I20260811 09:03:31.071787 23004 raft_consensus.cc:2229] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:03:31.072361 23004 raft_consensus.cc:2258] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:03:31.072681 22389 catalog_manager.cc:4841] TS 4d8d6ffdd1d643518f0b2aa5d36c032e (127.9.40.65:38795): tablet dab6cb559387496da9a43a5526c6ea07 (table test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]) successfully deleted
I20260811 09:03:31.073354 23006 tablet_replica.cc:330] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: stopping tablet replica
I20260811 09:03:31.073787 23006 raft_consensus.cc:2229] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:03:31.074291 23006 raft_consensus.cc:2258] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:03:31.075587 23004 ts_tablet_manager.cc:1905] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:03:31.076350 23006 ts_tablet_manager.cc:1905] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:03:31.083139 23004 ts_tablet_manager.cc:1918] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:03:31.083309 23004 log.cc:1196] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/dab6cb559387496da9a43a5526c6ea07
I20260811 09:03:31.083952 23004 ts_tablet_manager.cc:1939] T dab6cb559387496da9a43a5526c6ea07 P fb8191fd6d324abca7f74d0782bf6f72: Deleting consensus metadata
I20260811 09:03:31.085247 23006 ts_tablet_manager.cc:1918] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:03:31.085511 23006 log.cc:1196] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/dab6cb559387496da9a43a5526c6ea07
I20260811 09:03:31.086179 22389 catalog_manager.cc:4841] TS fb8191fd6d324abca7f74d0782bf6f72 (127.9.40.67:39671): tablet dab6cb559387496da9a43a5526c6ea07 (table test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]) successfully deleted
I20260811 09:03:31.086396 23006 ts_tablet_manager.cc:1939] T dab6cb559387496da9a43a5526c6ea07 P 0fea02af4163446092de2c7e3e72514a: Deleting consensus metadata
Aug 11 09:03:31 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:03:31.088567 22388 catalog_manager.cc:4841] TS 0fea02af4163446092de2c7e3e72514a (127.9.40.66:42705): tablet dab6cb559387496da9a43a5526c6ea07 (table test-table [id=32ac39a4fff94aa1b3c5fcc5cd4bc423]) successfully deleted
I20260811 09:03:31.096568 22978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:31.052924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42306 (local address 127.9.40.65:38795)
0811 09:03:31.053206 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:31.053217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:31.068769 (+ 15552us) server_negotiation.cc:407] Connection header received
0811 09:03:31.069495 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:31.069506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:31.069639 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:31.069764 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:31.074432 (+ 4668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.075906 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.078357 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.079346 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.079543 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:31.088045 (+ 8502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:31.088071 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:31.088090 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:31.088134 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:31.093068 (+ 4934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.093986 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.093993 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.093996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.094277 (+ 281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.094706 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.094713 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.094715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.095180 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:31.095497 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:31.095809 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:03:31.096027 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 09:03:31.097694 22646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:03:31.199864 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 22497
I20260811 09:03:31.224287 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 22665
Aug 11 09:03:31 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:03:31.240088 22976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:31.225326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46174 (local address 127.9.40.66:42705)
0811 09:03:31.225600 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:31.225605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:31.225621 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:03:31.225687 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:31.225690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:31.225758 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:31.225867 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:31.227198 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.227980 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.229411 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.229795 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.229917 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:31.234148 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:31.234162 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:31.234179 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:31.234212 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:31.237311 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.238002 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.238015 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.238017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.238152 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.238538 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.238544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.238547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.238887 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:31.239200 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:31.239505 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 09:03:31.239747 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 09:03:31.241228 22807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:03:31.332844 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 22665
I20260811 09:03:31.355849 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 22820
Aug 11 09:03:31 dist-test-slave-cbsf krb5kdc[21872](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:03:31.370867 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:03:31.356607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.9.40.67:39671)
0811 09:03:31.356837 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 09:03:31.356845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:03:31.356871 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:03:31.356951 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:03:31.356955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:03:31.357017 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:03:31.357101 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:03:31.358400 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.359146 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.360368 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:03:31.360939 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:03:31.361032 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:03:31.365559 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:03:31.365580 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:03:31.365598 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:03:31.365629 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:03:31.368429 (+ 2800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.369084 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.369091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.369093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.369215 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:03:31.369525 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:03:31.369528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:03:31.369530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:03:31.369885 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:03:31.370113 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:03:31.370309 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 09:03:31.370466 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 09:03:31.371568 22951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:03:31.462218 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 22820
I20260811 09:03:31.484166 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 22368
I20260811 09:03:31.485351 22441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:03:31.605413 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 22368
I20260811 09:03:31.636876 9377 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:03:31.981694 9377 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:03:31.981953 9377 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:03:32.339224 9377 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:03:32.339 UTC [21904] LOG: received smart shutdown request
2026-08-11 09:03:32.344 UTC [21904] LOG: background worker "logical replication launcher" (PID 21913) exited with exit code 1
2026-08-11 09:03:32.345 UTC [21908] LOG: shutting down
2026-08-11 09:03:32.484 UTC [21904] LOG: database system is shut down
2026-08-11T09:03:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54196 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:03:32 dist-test-slave-cbsf krb5kdc[23024](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:03:32 dist-test-slave-cbsf krb5kdc[23024](info): set up 2 sockets
Aug 11 09:03:32 dist-test-slave-cbsf krb5kdc[23024](info): commencing operation
krb5kdc: starting...
W20260811 09:03:33.823496 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:03:34 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:03:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:03:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
I20260811 09:03:34.216614 9377 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskSMwUEY/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/postgres -l logfile start
2026-08-11 09:03:36.711 UTC [23056] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:03:36.711 UTC [23056] LOG: listening on IPv4 address "127.9.40.84", port 37003
2026-08-11 09:03:36.714 UTC [23056] LOG: listening on Unix socket "/tmp/.s.PGSQL.37003"
2026-08-11 09:03:36.738 UTC [23059] LOG: database system was shut down at 2026-08-11 09:03:35 UTC
2026-08-11 09:03:36.742 UTC [23056] LOG: database system is ready to accept connections
I20260811 09:03:37.799548 9377 mini_postgres.cc:96] Postgres bound to 37003
2026-08-11 09:03:37.822 UTC [23069] FATAL: database "slave" does not exist
127.9.40.84:37003 - accepting connections
I20260811 09:03:37.823524 9377 mini_ranger.cc:161] Starting Ranger...
I20260811 09:03:37.853545 9377 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:03:38.347558 9377 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:03:38.347671 9377 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin
2026-08-11 09:03:38,649 [I] DB FLAVOR :POSTGRES
2026-08-11 09:03:38,650 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:03:38,650 [I] Checking connection..
2026-08-11 09:03:38,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:03:38,914 [I] Checking connection passed.
2026-08-11 09:03:38,914 [I] --------- Verifying version history table ---------
2026-08-11 09:03:38,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:03:39,199 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:03:39,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:03:39,487 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:03:39,487 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:03:39,487 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:03:39.760 UTC [23148] WARNING: there is no transaction in progress
2026-08-11 09:03:39,768 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:03:39,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:03:40,035 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:03:40,035 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:03:40,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:03:40,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:03:40,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-11 09:03:40,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:03:41,038 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:03:41,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:03:41,304 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:03:41,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:03:41,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:03:41,793 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:03:41,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:03:42.780 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.796 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.813 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.828 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.844 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.862 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.923 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.930 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.945 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.955 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.966 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.981 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:42.992 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:43.001 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:43.008 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:43.016 UTC [23275] WARNING: there is no transaction in progress
2026-08-11 09:03:43,667 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:03:43,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:03:43,923 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:03:43,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:03:44,188 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:03:44,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:03:44,451 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:03:44,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:03:44,693 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:03:44,958 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-11 09:03:45,233 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:03:45,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:03:45,493 [I] DB_PATCHES have already been applied
I20260811 09:03:45.499711 9377 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskSMwUEY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:03:45.499830 9377 mini_ranger.cc:193] Using host: 127.9.40.84
I20260811 09:03:45.519767 9377 mini_ranger.cc:227] Ranger admin URL: http://127.9.40.84:43205
Aug 11, 2026 9:03:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:03:45 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:03:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:03:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:03:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:03:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:03:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:03:46.762212 9377 mini_ranger.cc:160] Time spent starting Ranger: real 8.939s user 0.000s sys 0.055s
Aug 11, 2026 9:04:07 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:04:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 9:04:07 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:04:07 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:04:08 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:04:09.152129 9377 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:04:09 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439049, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:04:09.346205 9377 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:04:09.370101 9377 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:04:10.218799 9377 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:04:10.218931 9377 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms
I20260811 09:04:10.218986 9377 mini_ranger_kms.cc:227] Using postgres at 127.9.40.84:37003
2026-08-11 09:04:10,600 [I] DB FLAVOR :POSTGRES
2026-08-11 09:04:10,600 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:04:10,600 [I] Checking connection
2026-08-11 09:04:10,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:04:10,831 [I] connection success
2026-08-11 09:04:10,831 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:04:10,831 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:04:10,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:04:11,072 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:04:11,072 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:04:11,072 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:04:11,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:37003/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:04:11,353 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:04:11.831864 9377 mini_ranger_kms.cc:310] Created kms service
I20260811 09:04:12.035768 9377 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:04:12.082580 9377 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:04:12 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439049, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:12.496518 9377 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:04:12.496870 9377 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms:/tmp/dist-test-taskSMwUEY/build/asan/bin/postgresql.jar:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskSMwUEY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:04:12.496928 9377 mini_ranger_kms.cc:240] Using host: 127.9.40.84
I20260811 09:04:12.524180 9377 mini_ranger_kms.cc:276] Ranger KMS PID: 23482
I20260811 09:04:12.524363 9377 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.40.84:35789
Aug 11, 2026 9:04:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:04:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:04:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:04:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:04:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:04:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:04:14 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439054, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:15.038800 9377 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.693s user 0.066s sys 0.348s
I20260811 09:04:15.039114 9377 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:04:15.039331 9377 mini_ranger_kms.cc:401] 127.9.40.84:35789/kms/v1/keys
Aug 11 09:04:15 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439054, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:04:15 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439054, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:04:18 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:04:18.216446 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46647
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46647
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:18.728518 23535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:18.728821 23535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:18.728988 23535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:18.736304 23535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:18.736424 23535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:18.736546 23535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:18.736649 23535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:18.736737 23535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:18.747314 23535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.40.84:35789/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46647
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:18.748720 23535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:18.750099 23535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:18.762233 23540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:18.762274 23543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:18.762233 23541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:18.763265 23535 server_base.cc:1034] running on GCE node
I20260811 09:04:18.764029 23535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:18.765465 23535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:18.766697 23535 hybrid_clock.cc:648] HybridClock initialized: now 1786439058766669 us; error 46 us; skew 500 ppm
I20260811 09:04:18.767055 23535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:18 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:18.770255 23535 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:18.771829 23535 webserver.cc:460] Webserver started at http://127.9.40.126:37491/ using document root <none> and password file <none>
I20260811 09:04:18.772485 23535 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:18.772631 23535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:18.772975 23535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:04:18 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:18.833256 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "a8c23f198c334591a1bf1bd5826dfeae"
format_stamp: "Formatted at 2026-08-11 09:04:18 on dist-test-slave-cbsf"
server_key: "f6e99121eb0d219f82debe93211f556a"
server_key_iv: "86566fdb3f202bf44203f63098fda418"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:18.834227 23535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "a8c23f198c334591a1bf1bd5826dfeae"
format_stamp: "Formatted at 2026-08-11 09:04:18 on dist-test-slave-cbsf"
server_key: "f6e99121eb0d219f82debe93211f556a"
server_key_iv: "86566fdb3f202bf44203f63098fda418"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:18.839917 23535 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:04:18.861338 23551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:18.862902 23535 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:04:18.863123 23535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "a8c23f198c334591a1bf1bd5826dfeae"
format_stamp: "Formatted at 2026-08-11 09:04:18 on dist-test-slave-cbsf"
server_key: "f6e99121eb0d219f82debe93211f556a"
server_key_iv: "86566fdb3f202bf44203f63098fda418"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:18.863394 23535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:18.878708 23535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:18.882795 23535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:18.883152 23535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:18.903136 23535 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46647
I20260811 09:04:18.903190 23610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46647 every 8 connection(s)
I20260811 09:04:18.904592 23535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:18.908408 23613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:18.914525 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 23535
I20260811 09:04:18.914778 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:04:18.915473 9377 external_mini_cluster.cc:1428] Setting key dcc3bb0bc1270bb5a8f494b90b357f40
I20260811 09:04:18.919920 23613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Bootstrap starting.
I20260811 09:04:18.923707 23613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:18.924801 23613 log.cc:824] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:18.927940 23613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: No bootstrap required, opened a new log
Aug 11 09:04:18 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:18.936347 23613 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:18.936733 23613 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:18.936846 23613 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c23f198c334591a1bf1bd5826dfeae, State: Initialized, Role: FOLLOWER
I20260811 09:04:18.937522 23613 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:18.937784 23613 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:18.937948 23613 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:18.938167 23613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:18.939661 23616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:18.917964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.9.40.126:46647)
0811 09:04:18.918806 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:18.918819 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:18.918862 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:04:18.920730 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:18.920766 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:18.921146 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:18.921482 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:18.923250 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:18.924801 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:18.926458 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:18.927141 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:18.927266 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:18.932107 (+ 4841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:18.932115 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:18.932131 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:18.932166 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:18.935511 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:18.936213 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:18.936220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:18.936226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:18.936373 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:18.936836 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:18.936841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:18.936846 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:18.937426 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:18.937744 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:18.938221 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 09:04:18.938635 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":393,"threads_started":1}
I20260811 09:04:18.941131 23613 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:18.941632 23613 leader_election.cc:302] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c23f198c334591a1bf1bd5826dfeae; no voters:
I20260811 09:04:18.942242 23613 leader_election.cc:288] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:04:18.942399 23619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:18.943626 23619 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 LEADER]: Becoming Leader. State: Replica: a8c23f198c334591a1bf1bd5826dfeae, State: Running, Role: LEADER
I20260811 09:04:18.944095 23619 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:18.944515 23613 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:18.947504 23621 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c23f198c334591a1bf1bd5826dfeae. Latest consensus state: current_term: 1 leader_uuid: "a8c23f198c334591a1bf1bd5826dfeae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } } }
I20260811 09:04:18.947849 23621 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:18.948808 23625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:18.948662 23620 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8c23f198c334591a1bf1bd5826dfeae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } } }
I20260811 09:04:18.949002 23620 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:18.954298 23625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:18.965975 23625 catalog_manager.cc:1347] Generated new cluster ID: 4514bcab1e2c499bb0d0dc34b1cb7a32
I20260811 09:04:18.966107 23625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:18.969651 23613 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:04:18.988447 23625 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:04:18.990144 23625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:19.007117 23625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Generated new TSK 0
I20260811 09:04:19.008041 23625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:04:19.829943 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:04:20.287063 23664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:20.287599 23664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:20.287833 23664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:20.301311 23664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:20.301470 23664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:20.301741 23664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:04:20.319808 23664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:20.322031 23664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:20.323993 23664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:20.347251 23676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:20.348141 23677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:20.351773 23680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:20.352818 23664 server_base.cc:1034] running on GCE node
I20260811 09:04:20.353756 23664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:20.355082 23664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:20.356420 23664 hybrid_clock.cc:648] HybridClock initialized: now 1786439060356344 us; error 60 us; skew 500 ppm
I20260811 09:04:20.356855 23664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:20.361299 23664 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:04:20.363626 23664 webserver.cc:460] Webserver started at http://127.9.40.65:45729/ using document root <none> and password file <none>
I20260811 09:04:20.364665 23664 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:20.364856 23664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:20.365250 23664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:20.388942 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "0fe5728065a04ea38e2e2390f50b533a"
format_stamp: "Formatted at 2026-08-11 09:04:20 on dist-test-slave-cbsf"
server_key: "9b01b60d2ad3215bea4440a3791b4be0"
server_key_iv: "ca977a8bfe3098646d5db71da10cd84d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:20.390219 23664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "0fe5728065a04ea38e2e2390f50b533a"
format_stamp: "Formatted at 2026-08-11 09:04:20 on dist-test-slave-cbsf"
server_key: "9b01b60d2ad3215bea4440a3791b4be0"
server_key_iv: "ca977a8bfe3098646d5db71da10cd84d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:20.398018 23664 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 09:04:20.434533 23689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:20.437654 23664 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.005s sys 0.008s
I20260811 09:04:20.437969 23664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "0fe5728065a04ea38e2e2390f50b533a"
format_stamp: "Formatted at 2026-08-11 09:04:20 on dist-test-slave-cbsf"
server_key: "9b01b60d2ad3215bea4440a3791b4be0"
server_key_iv: "ca977a8bfe3098646d5db71da10cd84d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:20.438366 23664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:20.471724 23664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:20.477258 23664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:20.477694 23664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:20.479038 23664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:20.487232 23664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:20.487381 23664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:20.487608 23664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:20.487713 23664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:20.573971 23664 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42283
I20260811 09:04:20.574074 23836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42283 every 8 connection(s)
I20260811 09:04:20.575822 23664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:04:20.577801 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 23664
I20260811 09:04:20.578023 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:04:20.578786 9377 external_mini_cluster.cc:1428] Setting key b12b9c2700f90b71c06e6a89533161ca
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:20.624015 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:20.594531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33921 (local address 127.9.40.126:46647)
0811 09:04:20.602511 (+ 7980us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:20.602520 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:20.602546 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:04:20.602641 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:20.602649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:20.602769 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:20.602909 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:20.606409 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:20.607589 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:20.610119 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:20.610786 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:20.610884 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:20.616435 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:20.616442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:20.616447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:20.616501 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:20.620190 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:20.621193 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:20.621199 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:20.621202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:20.621353 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:20.621886 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:20.621891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:20.621895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:20.622246 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:20.622374 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:20.623374 (+ 1000us) server_negotiation.cc:299] Negotiation successful
0811 09:04:20.623620 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7805,"thread_start_us":7687,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260811 09:04:20.633790 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:20.593843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.65:33921)
0811 09:04:20.598354 (+ 4511us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:20.598413 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:20.601670 (+ 3257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:20.604288 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:20.604302 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:20.605078 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:20.605990 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:20.606019 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:20.607805 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:20.607811 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:20.609925 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:20.609947 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:20.611251 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:20.611258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:20.611560 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:20.612373 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:20.612402 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:20.616154 (+ 3752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:20.620382 (+ 4228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:20.620391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:20.620409 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:20.621048 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:20.621493 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:20.621498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:20.621501 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:20.621750 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:20.622471 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:20.622479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:20.622921 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 09:04:20.624602 (+ 1681us) client_negotiation.cc:240] Negotiation successful
0811 09:04:20.625198 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4192,"thread_start_us":3950,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
I20260811 09:04:20.636847 23837 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:20.637210 23837 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:20.638059 23837 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:20.646647 23570 ts_manager.cc:194] Registered new tserver with Master: 0fe5728065a04ea38e2e2390f50b533a (127.9.40.65:42283)
I20260811 09:04:20.649122 23570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:33921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
W20260811 09:04:20.687157 9377 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.9.40.66: real 0.105s user 0.002s sys 0.026s
I20260811 09:04:20.688082 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:20 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439060, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
W20260811 09:04:21.201373 23844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:21.201900 23844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:21.202124 23844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:21.214104 23844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:21.214346 23844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:21.214582 23844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:04:21.229668 23844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:21.231647 23844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:21.233501 23844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:21.283823 23852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:21.287374 23851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:21.298919 23855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:21.299521 23844 server_base.cc:1034] running on GCE node
I20260811 09:04:21.300830 23844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:21.306679 23844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:21.308295 23844 hybrid_clock.cc:648] HybridClock initialized: now 1786439061308169 us; error 121 us; skew 500 ppm
I20260811 09:04:21.308869 23844 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:21 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:21.320768 23844 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:04:21.351326 23844 webserver.cc:460] Webserver started at http://127.9.40.66:33829/ using document root <none> and password file <none>
I20260811 09:04:21.352756 23844 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:21.352979 23844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:21.353361 23844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:04:21 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:21.375247 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "cc22b5b42f17469f96afa45eba363825"
format_stamp: "Formatted at 2026-08-11 09:04:21 on dist-test-slave-cbsf"
server_key: "d3d445a7ec0cfd037dbd59322118b59b"
server_key_iv: "1cb1f4f0784e679a2be7baf270407a5d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:21.376212 23844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "cc22b5b42f17469f96afa45eba363825"
format_stamp: "Formatted at 2026-08-11 09:04:21 on dist-test-slave-cbsf"
server_key: "d3d445a7ec0cfd037dbd59322118b59b"
server_key_iv: "1cb1f4f0784e679a2be7baf270407a5d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:21.382282 23844 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:04:21.429474 23870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:21.431828 23844 fs_manager.cc:725] Time spent opening block manager: real 0.030s user 0.014s sys 0.013s
I20260811 09:04:21.432152 23844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "cc22b5b42f17469f96afa45eba363825"
format_stamp: "Formatted at 2026-08-11 09:04:21 on dist-test-slave-cbsf"
server_key: "d3d445a7ec0cfd037dbd59322118b59b"
server_key_iv: "1cb1f4f0784e679a2be7baf270407a5d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:21.432582 23844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:21.528296 23844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:21.534868 23844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:21.535409 23844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:21.543196 23844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:21.551832 23844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:21.552071 23844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:04:21.552282 23844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:21.552400 23844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:21.654043 23837 heartbeater.cc:503] Master 127.9.40.126:46647 was elected leader, sending a full tablet report...
I20260811 09:04:21.716841 23844 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:33585
I20260811 09:04:21.716866 23996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:33585 every 8 connection(s)
I20260811 09:04:21.718930 23844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:04:21.724848 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 23844
I20260811 09:04:21.725024 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:04:21.725586 9377 external_mini_cluster.cc:1428] Setting key f9fe6f8dc626d729579773180b329fb1
Aug 11 09:04:21 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439061, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:21.755877 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:21.723460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35875 (local address 127.9.40.126:46647)
0811 09:04:21.729194 (+ 5734us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:21.729202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:21.732756 (+ 3554us) server_negotiation.cc:407] Connection header received
0811 09:04:21.733439 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:21.733447 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:21.733548 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:21.733652 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:21.736320 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:21.737672 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:21.740473 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:21.741274 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:21.741397 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:21.746835 (+ 5438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:21.746843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:21.746849 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:21.746915 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:21.751127 (+ 4212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:21.752367 (+ 1240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:21.752375 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:21.752379 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:21.752566 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:21.753268 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:21.753275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:21.753279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:21.753725 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:21.753882 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:21.755105 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 09:04:21.755391 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5574,"thread_start_us":9914,"threads_started":1}
I20260811 09:04:21.759239 23999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:21.722808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.66:35875)
0811 09:04:21.729925 (+ 7117us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:21.730004 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:21.733031 (+ 3027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:21.734306 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:21.734320 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:21.735075 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:21.736060 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:21.736091 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:21.737950 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:21.737956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:21.740263 (+ 2307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:21.740281 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:21.741806 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:21.741814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:21.742159 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:21.743108 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:21.743147 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:21.746514 (+ 3367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:21.751356 (+ 4842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:21.751368 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:21.751390 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:21.752183 (+ 793us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:21.752789 (+ 606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:21.752794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:21.752799 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:21.753100 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:21.754028 (+ 928us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:21.754039 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:21.754550 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 09:04:21.756623 (+ 2073us) client_negotiation.cc:240] Negotiation successful
0811 09:04:21.757459 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6689,"thread_start_us":9494,"threads_started":1}
I20260811 09:04:21.761593 23997 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:21.761955 23997 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:21.762907 23997 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:21.764876 23570 ts_manager.cc:194] Registered new tserver with Master: cc22b5b42f17469f96afa45eba363825 (127.9.40.66:33585)
I20260811 09:04:21.766220 23570 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35875
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:04:21.813481 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:22.216842 24003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:22.217156 24003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:22.217257 24003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:22.224433 24003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:22.224555 24003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:22.224668 24003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:04:22.234716 24003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:22.236022 24003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:22.237401 24003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:22.250713 24008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:22.250972 24011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:22.250936 24009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:22.252029 24003 server_base.cc:1034] running on GCE node
I20260811 09:04:22.252650 24003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:22.253809 24003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:22.255059 24003 hybrid_clock.cc:648] HybridClock initialized: now 1786439062254985 us; error 48 us; skew 500 ppm
I20260811 09:04:22.255398 24003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:22.258941 24003 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:04:22.260438 24003 webserver.cc:460] Webserver started at http://127.9.40.67:41025/ using document root <none> and password file <none>
I20260811 09:04:22.261169 24003 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:22.261276 24003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:22.261538 24003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:22.279909 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c"
format_stamp: "Formatted at 2026-08-11 09:04:22 on dist-test-slave-cbsf"
server_key: "e59c19bf96f4e10da4e67440efe860c5"
server_key_iv: "b5fec8729e1115f96c9478f074f73e84"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:22.281104 24003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c"
format_stamp: "Formatted at 2026-08-11 09:04:22 on dist-test-slave-cbsf"
server_key: "e59c19bf96f4e10da4e67440efe860c5"
server_key_iv: "b5fec8729e1115f96c9478f074f73e84"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:22.288120 24003 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.003s
I20260811 09:04:22.306773 24018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:22.308271 24003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:04:22.308483 24003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c"
format_stamp: "Formatted at 2026-08-11 09:04:22 on dist-test-slave-cbsf"
server_key: "e59c19bf96f4e10da4e67440efe860c5"
server_key_iv: "b5fec8729e1115f96c9478f074f73e84"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:22.308748 24003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:22.346519 24003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:22.350339 24003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:22.350657 24003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:22.351640 24003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:22.353082 24003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:22.353184 24003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:22.353338 24003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:22.353412 24003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:22.387496 24003 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35417
I20260811 09:04:22.387542 24152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35417 every 8 connection(s)
I20260811 09:04:22.388895 24003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:04:22.394598 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24003
I20260811 09:04:22.394814 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:04:22.395431 9377 external_mini_cluster.cc:1428] Setting key cfb63395bcdecb278ecc5e6ac5c24aef
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439062, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:22.408493 24155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:22.391512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54183 (local address 127.9.40.126:46647)
0811 09:04:22.392037 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:22.392042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:22.393006 (+ 964us) server_negotiation.cc:407] Connection header received
0811 09:04:22.394862 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:22.394868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:22.394976 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:22.395117 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:22.396643 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.397477 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.399014 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.399448 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.399592 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:22.402839 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:22.402844 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:22.402847 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:22.402879 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:22.405423 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.406102 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.406109 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.406113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.406243 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.406709 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.406716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.406719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.407038 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:22.407198 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:22.407861 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 09:04:22.408110 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":254,"threads_started":1}
I20260811 09:04:22.409494 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:22.392064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.67:54183)
0811 09:04:22.392843 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:22.392880 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:22.394564 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:22.395310 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:22.395316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:22.395802 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:22.396450 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:22.396487 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.397650 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.397655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:22.398865 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:22.398877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.399698 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.399706 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:22.399936 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:22.400562 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:22.400583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:22.402615 (+ 2032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:22.405564 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:22.405572 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:22.405589 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:22.405987 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:22.406355 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:22.406360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:22.406363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:22.406567 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:22.407170 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:22.407177 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:22.407513 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 09:04:22.407778 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 09:04:22.408288 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":316,"threads_started":1}
I20260811 09:04:22.411171 24153 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:22.411497 24153 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:22.412212 24153 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:22.413756 23569 ts_manager.cc:194] Registered new tserver with Master: cf0b13ba9ddc45ebbc23cb02f55a397c (127.9.40.67:35417)
I20260811 09:04:22.414705 23569 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:54183
I20260811 09:04:22.423655 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:04:22.425369 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 23664
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:04:22.442332 24159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:22.426326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40946 (local address 127.9.40.65:42283)
0811 09:04:22.426984 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:22.426991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:22.427016 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:04:22.427074 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:22.427080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:22.427277 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:22.427409 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:22.428977 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.430043 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.431483 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.432067 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.432178 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:22.436844 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:22.436862 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:22.436883 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:22.436938 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:22.439863 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.440475 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.440481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.440483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.440642 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.440998 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.441001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.441003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.441328 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:22.441555 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:22.441806 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 09:04:22.442006 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":350,"threads_started":1}
I20260811 09:04:22.443616 23830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:22.527124 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 23664
I20260811 09:04:22.547190 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 23844
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:04:22.568961 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:22.548182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.9.40.66:33585)
0811 09:04:22.548812 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:22.548819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:22.548845 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:04:22.548907 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:22.548913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:22.549115 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:22.549255 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:22.550582 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.551627 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.553172 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.553808 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.553907 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:22.563045 (+ 9138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:22.563058 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:22.563073 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:22.563116 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:22.566002 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.566758 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.566766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.566768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.566927 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.567282 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.567286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.567288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.567636 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:22.567926 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:22.568246 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 09:04:22.568503 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":311,"threads_started":1}
I20260811 09:04:22.570317 23992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:22.652746 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 23844
I20260811 09:04:22.672297 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24003
Aug 11 09:04:22 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:04:22.689034 24164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:22.673337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.9.40.67:35417)
0811 09:04:22.674012 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:22.674020 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:22.674052 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:04:22.674115 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:22.674122 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:22.674365 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:22.674511 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:22.675750 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.676998 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.678269 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:22.678833 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:22.678931 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:22.683233 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:22.683245 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:22.683262 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:22.683316 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:22.686405 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.687100 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.687106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.687109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.687245 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:22.687583 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:22.687586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:22.687588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:22.687942 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:22.688189 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:22.688452 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 09:04:22.688667 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":355,"threads_started":1}
I20260811 09:04:22.690363 24146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:22.774642 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24003
I20260811 09:04:22.795220 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 23535
I20260811 09:04:22.796680 23605 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:22.905469 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 23535
I20260811 09:04:22.935242 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46647
--webserver_interface=127.9.40.126
--webserver_port=37491
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46647
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:23.281793 24168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:23.282097 24168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:23.282290 24168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:23.289467 24168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:23.289605 24168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:23.289693 24168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:23.289780 24168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:23.289853 24168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:23.300367 24168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46647
--ranger_config_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=37491
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:23.301797 24168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:23.303194 24168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:23.315495 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:23.315552 24175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:23.315563 24176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:23.316965 24168 server_base.cc:1034] running on GCE node
I20260811 09:04:23.317818 24168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:23.319243 24168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:23.320518 24168 hybrid_clock.cc:648] HybridClock initialized: now 1786439063320484 us; error 44 us; skew 500 ppm
I20260811 09:04:23.320945 24168 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:23 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:23.324220 24168 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:23.325766 24168 webserver.cc:460] Webserver started at http://127.9.40.126:37491/ using document root <none> and password file <none>
I20260811 09:04:23.326396 24168 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:23.326524 24168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:04:23 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439063, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:23.406962 24168 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:04:23.410203 24187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:23.411645 24168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:04:23.411978 24168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "a8c23f198c334591a1bf1bd5826dfeae"
format_stamp: "Formatted at 2026-08-11 09:04:18 on dist-test-slave-cbsf"
server_key: "f6e99121eb0d219f82debe93211f556a"
server_key_iv: "86566fdb3f202bf44203f63098fda418"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:23.413060 24168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:23.433606 24168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:23.438014 24168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:23.438277 24168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:23.457710 24168 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46647
I20260811 09:04:23.457717 24254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46647 every 8 connection(s)
I20260811 09:04:23.459312 24168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:23.465466 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24168
I20260811 09:04:23.466589 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:42283
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:46647
--webserver_port=45729
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:23.467100 24255 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:04:23.472517 24255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Bootstrap starting.
I20260811 09:04:23.493863 24255 log.cc:824] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:23.502998 24255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:04:23.503597 24255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Bootstrap complete.
I20260811 09:04:23.513422 24255 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:23.514328 24255 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8c23f198c334591a1bf1bd5826dfeae, State: Initialized, Role: FOLLOWER
I20260811 09:04:23.514994 24255 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:23.515239 24255 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:23.515380 24255 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:23.515581 24255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:04:23.519995 24255 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:23.520541 24255 leader_election.cc:302] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8c23f198c334591a1bf1bd5826dfeae; no voters:
I20260811 09:04:23.521216 24255 leader_election.cc:288] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:04:23.521354 24259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:04:23.522469 24259 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [term 2 LEADER]: Becoming Leader. State: Replica: a8c23f198c334591a1bf1bd5826dfeae, State: Running, Role: LEADER
I20260811 09:04:23.523141 24259 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } }
I20260811 09:04:23.523483 24255 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:23.526096 24261 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8c23f198c334591a1bf1bd5826dfeae. Latest consensus state: current_term: 2 leader_uuid: "a8c23f198c334591a1bf1bd5826dfeae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } } }
I20260811 09:04:23.526134 24260 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a8c23f198c334591a1bf1bd5826dfeae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8c23f198c334591a1bf1bd5826dfeae" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46647 } } }
I20260811 09:04:23.526733 24260 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:23.526779 24261 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:23.527908 24264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:23.541749 24264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:23.543746 24264 catalog_manager.cc:1259] Loaded cluster ID: 4514bcab1e2c499bb0d0dc34b1cb7a32
I20260811 09:04:23.543958 24264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:23.549497 24264 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:23.552076 24264 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Loaded TSK: 0
I20260811 09:04:23.554111 24264 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:04:23.560236 24255 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 09:04:23.965580 24257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:23.966049 24257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:23.966274 24257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:23.979712 24257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:23.979940 24257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:23.980201 24257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:04:23.997869 24257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:42283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=45729
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:23.999740 24257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:24.001561 24257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:24.049288 24294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:24.050904 24295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:24.057706 24299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:24.072623 24257 server_base.cc:1034] running on GCE node
I20260811 09:04:24.073789 24257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:24.083781 24257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:24.085198 24257 hybrid_clock.cc:648] HybridClock initialized: now 1786439064085126 us; error 74 us; skew 500 ppm
I20260811 09:04:24.085687 24257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:24.094079 24257 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:04:24.125790 24257 webserver.cc:460] Webserver started at http://127.9.40.65:45729/ using document root <none> and password file <none>
I20260811 09:04:24.126916 24257 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:24.127130 24257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:24.157740 24257 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.006s sys 0.000s
I20260811 09:04:24.167483 24307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:24.170291 24257 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.009s sys 0.000s
I20260811 09:04:24.170588 24257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "0fe5728065a04ea38e2e2390f50b533a"
format_stamp: "Formatted at 2026-08-11 09:04:20 on dist-test-slave-cbsf"
server_key: "9b01b60d2ad3215bea4440a3791b4be0"
server_key_iv: "ca977a8bfe3098646d5db71da10cd84d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:24.172122 24257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:24.202812 24257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:24.209266 24257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:24.209653 24257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:24.211047 24257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:24.213827 24257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:24.213997 24257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:04:24.214159 24257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:24.214283 24257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:24.282248 24257 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42283
I20260811 09:04:24.282315 24433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42283 every 8 connection(s)
I20260811 09:04:24.284485 24257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:04:24.286592 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24257
I20260811 09:04:24.287464 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:33585
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:46647
--webserver_port=33829
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:24.339993 24436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:24.288925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.65:54203)
0811 09:04:24.310887 (+ 21962us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:24.310956 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:24.313568 (+ 2612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:24.315859 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:24.315874 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:24.316608 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:24.317683 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:24.317715 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:24.320398 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:24.320405 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:24.322769 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:24.322786 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:24.324502 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:24.324512 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:24.324887 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:24.325877 (+ 990us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:24.325913 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:24.329241 (+ 3328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:24.334527 (+ 5286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:24.334539 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:24.334557 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:24.335277 (+ 720us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:24.335914 (+ 637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:24.335920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:24.335924 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:24.336189 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:24.337353 (+ 1164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:24.337362 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:24.337775 (+ 413us) client_negotiation.cc:769] Sending connection context
0811 09:04:24.338061 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 09:04:24.338629 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21620,"thread_start_us":21378,"threads_started":1}
I20260811 09:04:24.342286 24437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:24.291922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:54203 (local address 127.9.40.126:46647)
0811 09:04:24.309542 (+ 17620us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:24.309566 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:24.311569 (+ 2003us) server_negotiation.cc:407] Connection header received
0811 09:04:24.314243 (+ 2674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:24.314263 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:24.314765 (+ 502us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:24.315149 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:24.317963 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:24.320178 (+ 2215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:24.322973 (+ 2795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:24.324207 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:24.324382 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:24.329606 (+ 5224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:24.329622 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:24.329649 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:24.329714 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:24.334296 (+ 4582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:24.335489 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:24.335499 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:24.335507 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:24.335768 (+ 261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:24.336354 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:24.336360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:24.336364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:24.337051 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:24.337273 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:24.340679 (+ 3406us) server_negotiation.cc:299] Negotiation successful
0811 09:04:24.341109 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17276,"thread_start_us":17509,"threads_started":1}
I20260811 09:04:24.345989 24434 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:24.346307 24434 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:24.347193 24434 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
W20260811 09:04:24.350701 24434 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:46647 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:04:24.350955 24434 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:24.351260 24434 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:24.352504 24434 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:24.352878 24434 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
W20260811 09:04:24.353978 24434 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:04:24.770105 24439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:24.770383 24439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:24.770527 24439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:24.782310 24439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:24.782531 24439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:24.782773 24439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:04:24.798735 24439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:33585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=33829
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:24.800546 24439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:24.802352 24439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:24.821261 24461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:24.825136 24460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:24.826683 24465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:24.827461 24439 server_base.cc:1034] running on GCE node
I20260811 09:04:24.828186 24439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:24.829509 24439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:24.830770 24439 hybrid_clock.cc:648] HybridClock initialized: now 1786439064830690 us; error 64 us; skew 500 ppm
I20260811 09:04:24.831177 24439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:24.835395 24439 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:04:24.837453 24439 webserver.cc:460] Webserver started at http://127.9.40.66:33829/ using document root <none> and password file <none>
I20260811 09:04:24.838375 24439 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:24.838567 24439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 11 09:04:24 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:24.879496 24439 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.000s sys 0.012s
I20260811 09:04:24.893328 24473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:24.895284 24439 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.000s sys 0.012s
I20260811 09:04:24.895546 24439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "cc22b5b42f17469f96afa45eba363825"
format_stamp: "Formatted at 2026-08-11 09:04:21 on dist-test-slave-cbsf"
server_key: "d3d445a7ec0cfd037dbd59322118b59b"
server_key_iv: "1cb1f4f0784e679a2be7baf270407a5d"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:24.897053 24439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:24.931783 24439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:24.937352 24439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:24.937690 24439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:24.939028 24439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:24.940948 24439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:24.941062 24439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:24.941215 24439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:24.941298 24439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:24.986706 24439 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:33585
I20260811 09:04:24.986753 24603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:33585 every 8 connection(s)
I20260811 09:04:24.988570 24439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:04:24.990212 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24439
I20260811 09:04:24.991106 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:35789/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:35417
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:46647
--webserver_port=41025
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:25.048635 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:24.997717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56153 (local address 127.9.40.126:46647)
0811 09:04:25.004768 (+ 7051us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.004776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.004797 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:04:25.004867 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.004872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.004962 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.005080 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:25.012715 (+ 7635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.013892 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.016479 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.017165 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.017282 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.023089 (+ 5807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:25.023098 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:25.023105 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:25.023155 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:25.038725 (+ 15570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.043366 (+ 4641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.043376 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.043381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.043602 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.044379 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.044387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.044391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.044908 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:25.045053 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.047831 (+ 2778us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.048158 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6882,"thread_start_us":6780,"threads_started":1}
I20260811 09:04:25.051055 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:24.997076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.66:56153)
0811 09:04:25.000888 (+ 3812us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:25.000938 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:25.003965 (+ 3027us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:25.010066 (+ 6101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:25.010096 (+ 30us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:25.010753 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:25.011636 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.011664 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.014108 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.014113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.016286 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.016301 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.017657 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.017667 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.017985 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.018829 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:25.018863 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:25.022754 (+ 3891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:25.042420 (+ 19666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:25.042431 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:25.042449 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:25.043176 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:25.043822 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:25.043830 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:25.043837 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:25.044195 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:25.045188 (+ 993us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:25.045196 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:25.045729 (+ 533us) client_negotiation.cc:769] Sending connection context
0811 09:04:25.049176 (+ 3447us) client_negotiation.cc:240] Negotiation successful
0811 09:04:25.049776 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3503,"thread_start_us":3229,"threads_started":1}
I20260811 09:04:25.052951 24604 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:25.053303 24604 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:25.054165 24604 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:25.056376 24210 ts_manager.cc:194] Registered new tserver with Master: cc22b5b42f17469f96afa45eba363825 (127.9.40.66:33585)
I20260811 09:04:25.058878 24210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:56153
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:25.354135 24434 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:25.354573 24434 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:25.355904 24210 ts_manager.cc:194] Registered new tserver with Master: 0fe5728065a04ea38e2e2390f50b533a (127.9.40.65:42283)
I20260811 09:04:25.357208 24210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:54203
W20260811 09:04:25.455663 24608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:25.456009 24608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:25.456171 24608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:25.467494 24608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:25.467643 24608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:25.467877 24608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:04:25.478456 24608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:35789/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:35417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=41025
--tserver_master_addrs=127.9.40.126:46647
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:25.480406 24608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:25.482295 24608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:25.499439 24617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:25.499927 24616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:25.507730 24621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:25.509023 24608 server_base.cc:1034] running on GCE node
I20260811 09:04:25.509886 24608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:25.511229 24608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:25.512555 24608 hybrid_clock.cc:648] HybridClock initialized: now 1786439065512475 us; error 74 us; skew 500 ppm
I20260811 09:04:25.513024 24608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:25.517478 24608 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:04:25.519443 24608 webserver.cc:460] Webserver started at http://127.9.40.67:41025/ using document root <none> and password file <none>
I20260811 09:04:25.520327 24608 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:25.520473 24608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:25.555351 24608 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.010s sys 0.000s
I20260811 09:04:25.559443 24628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:25.561391 24608 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:04:25.561626 24608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c"
format_stamp: "Formatted at 2026-08-11 09:04:22 on dist-test-slave-cbsf"
server_key: "e59c19bf96f4e10da4e67440efe860c5"
server_key_iv: "b5fec8729e1115f96c9478f074f73e84"
server_key_version: "kuduclusterkey@0"
I20260811 09:04:25.563113 24608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:25.588630 24608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:25.593957 24608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:25.594295 24608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:25.595587 24608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:25.597487 24608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:25.597608 24608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:25.597728 24608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:25.597810 24608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439064, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260811 09:04:25.706838 24761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35417 every 8 connection(s)
I20260811 09:04:25.707265 24608 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35417
I20260811 09:04:25.709416 24608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:04:25.716135 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24608
Aug 11 09:04:25 dist-test-slave-cbsf krb5kdc[23024](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439065, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:25.777041 24767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.717687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55592 (local address 127.9.40.126:46647)
0811 09:04:25.747242 (+ 29555us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.747248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.747265 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:04:25.747321 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.747325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.747445 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.747566 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:25.752184 (+ 4618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.753155 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.754555 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.755350 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.755481 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.764985 (+ 9504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:25.764991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:25.764996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:25.765059 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:25.768884 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.774628 (+ 5744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.774632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.774635 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.774773 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.775277 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.775282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.775286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.775684 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:25.775946 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.776355 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.776629 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29437,"thread_start_us":32414,"threads_started":1}
I20260811 09:04:25.782918 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.719406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46647 (local address 127.9.40.67:45701)
0811 09:04:25.743983 (+ 24577us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:25.744062 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:25.746436 (+ 2374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:25.750914 (+ 4478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:25.750927 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:25.751638 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:25.753211 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.754595 (+ 1384us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.757079 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.757084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.758668 (+ 1584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.758682 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.759587 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.759595 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.759930 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.762192 (+ 2262us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:25.762226 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:25.769158 (+ 6932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:25.773537 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:25.773547 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:25.773568 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:25.774218 (+ 650us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:25.774989 (+ 771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:25.774995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:25.774998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:25.776095 (+ 1097us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:25.780029 (+ 3934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:25.780039 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:25.780514 (+ 475us) client_negotiation.cc:769] Sending connection context
0811 09:04:25.780876 (+ 362us) client_negotiation.cc:240] Negotiation successful
0811 09:04:25.781487 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24137,"thread_start_us":28190,"threads_started":1}
I20260811 09:04:25.784016 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.720046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45701 (local address 127.9.40.126:46647)
0811 09:04:25.749135 (+ 29089us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.749141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.749161 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:04:25.749215 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.749220 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.749306 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.749414 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:25.754821 (+ 5407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.756909 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.758886 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.759321 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.759457 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.769490 (+ 10033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:25.769498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:25.769503 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:25.769550 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:25.773361 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.774393 (+ 1032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.774398 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.774401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.774565 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.778188 (+ 3623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.778193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.778196 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.778546 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:25.778698 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.783420 (+ 4722us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.783665 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28950,"thread_start_us":28521,"threads_started":1}
I20260811 09:04:25.786262 24762 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46647
I20260811 09:04:25.786595 24762 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:25.787468 24762 heartbeater.cc:511] Master 127.9.40.126:46647 requested a full tablet report, sending...
I20260811 09:04:25.789235 24210 ts_manager.cc:194] Registered new tserver with Master: cf0b13ba9ddc45ebbc23cb02f55a397c (127.9.40.67:35417)
I20260811 09:04:25.790388 24210 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:45701
I20260811 09:04:25.793609 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:04:25.881981 24765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.859776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.9.40.126:46647)
0811 09:04:25.859983 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.860005 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.860031 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:04:25.860105 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.860112 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.860206 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.860317 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:25.862678 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.863878 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.865964 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.866753 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.866920 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.871225 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:25.871233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:25.871237 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:25.871285 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:25.878080 (+ 6795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.879309 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.879315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.879318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.879479 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:25.880039 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:25.880045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:25.880048 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:25.880451 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:25.880730 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.881368 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.881637 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:04:25.889885 24210 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:04:25.907176 24210 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:04:25.960909 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.937639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40962 (local address 127.9.40.65:42283)
0811 09:04:25.948414 (+ 10775us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.948430 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.950073 (+ 1643us) server_negotiation.cc:407] Connection header received
0811 09:04:25.950423 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.950431 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.950683 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.950877 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:25.952018 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.953965 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.957194 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.959105 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.959227 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.959459 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.960192 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.960436 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2150,"thread_start_us":833,"threads_started":1}
I20260811 09:04:25.961827 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.936790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42283 (local address 127.0.0.1:40962)
0811 09:04:25.949942 (+ 13152us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:25.949959 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:25.950210 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:25.951273 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:25.951280 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:25.951315 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:25.951771 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.951782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.954193 (+ 2411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.954202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.956936 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.956957 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.959672 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.959677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.959854 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.959923 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 09:04:25.961142 (+ 1219us) client_negotiation.cc:240] Negotiation successful
0811 09:04:25.961368 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13032,"thread_start_us":11856,"threads_started":1}
I20260811 09:04:25.962000 24778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.937017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58804 (local address 127.9.40.67:35417)
0811 09:04:25.937989 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.938000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.949315 (+ 11315us) server_negotiation.cc:407] Connection header received
0811 09:04:25.949609 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.949619 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.949907 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.950200 (+ 293us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:25.951289 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.953049 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.958814 (+ 5765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.960466 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.960584 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.960834 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.961491 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.961665 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":804,"threads_started":1}
I20260811 09:04:25.962713 24781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.936633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35417 (local address 127.0.0.1:58804)
0811 09:04:25.949120 (+ 12487us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:25.949162 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:25.949377 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:25.950597 (+ 1220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:25.950606 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:25.950638 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:25.951035 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.951050 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.953261 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.953267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.955713 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.955730 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.961054 (+ 5324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.961061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.961270 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.961307 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:04:25.962225 (+ 918us) client_negotiation.cc:240] Negotiation successful
0811 09:04:25.962371 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12200,"thread_start_us":14991,"threads_started":1}
I20260811 09:04:25.968080 24363 tablet_service.cc:1467] Processing CreateTablet for tablet d6f907eff86a43948c48216a5accee18 (DEFAULT_TABLE table=test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:25.970027 24363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f907eff86a43948c48216a5accee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:25.971158 24684 tablet_service.cc:1467] Processing CreateTablet for tablet d6f907eff86a43948c48216a5accee18 (DEFAULT_TABLE table=test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:25.972662 24684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f907eff86a43948c48216a5accee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:25.993543 24782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.952381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:33585 (local address 127.0.0.1:40934)
0811 09:04:25.962350 (+ 9969us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:25.962371 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:25.962491 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:25.981132 (+ 18641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:25.981138 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:25.981173 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:25.981401 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.981416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.988836 (+ 7420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.988843 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.990781 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:25.990802 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.992732 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.992739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:25.992959 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.993001 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 09:04:25.993090 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 09:04:25.993243 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9831,"thread_start_us":11700,"threads_started":1}
I20260811 09:04:25.995137 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:25.949918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.9.40.66:33585)
0811 09:04:25.963383 (+ 13465us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:25.963392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:25.963424 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:04:25.963534 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:25.963544 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:25.963841 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:25.981161 (+ 17320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:25.981679 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.983455 (+ 1776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.991043 (+ 7588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:25.992519 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:25.994222 (+ 1703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:25.994459 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:25.994534 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 09:04:25.994699 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13284,"thread_start_us":13622,"threads_started":1}
I20260811 09:04:25.997025 24788 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Bootstrap starting.
I20260811 09:04:25.999373 24528 tablet_service.cc:1467] Processing CreateTablet for tablet d6f907eff86a43948c48216a5accee18 (DEFAULT_TABLE table=test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:26.000927 24788 tablet_bootstrap.cc:654] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:26.001796 24528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d6f907eff86a43948c48216a5accee18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:26.003612 24788 log.cc:824] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:26.034956 24792 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Bootstrap starting.
I20260811 09:04:26.036023 24788 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: No bootstrap required, opened a new log
I20260811 09:04:26.036435 24788 ts_tablet_manager.cc:1397] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Time spent bootstrapping tablet: real 0.040s user 0.030s sys 0.008s
I20260811 09:04:26.047451 24788 raft_consensus.cc:348] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.048177 24788 raft_consensus.cc:374] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:26.048353 24788 raft_consensus.cc:729] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe5728065a04ea38e2e2390f50b533a, State: Initialized, Role: FOLLOWER
I20260811 09:04:26.049266 24788 consensus_queue.cc:260] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.049522 24794 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Bootstrap starting.
I20260811 09:04:26.050626 24792 tablet_bootstrap.cc:654] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:26.051012 24434 heartbeater.cc:503] Master 127.9.40.126:46647 was elected leader, sending a full tablet report...
I20260811 09:04:26.051779 24788 ts_tablet_manager.cc:1428] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 09:04:26.052099 24792 log.cc:824] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:26.053324 24794 tablet_bootstrap.cc:654] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:26.054636 24794 log.cc:824] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:26.062247 24792 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: No bootstrap required, opened a new log
I20260811 09:04:26.062659 24792 ts_tablet_manager.cc:1397] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Time spent bootstrapping tablet: real 0.029s user 0.008s sys 0.007s
I20260811 09:04:26.064368 24794 tablet_bootstrap.cc:492] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: No bootstrap required, opened a new log
I20260811 09:04:26.064646 24794 ts_tablet_manager.cc:1397] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260811 09:04:26.064925 24604 heartbeater.cc:503] Master 127.9.40.126:46647 was elected leader, sending a full tablet report...
I20260811 09:04:26.073746 24792 raft_consensus.cc:348] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.074307 24792 raft_consensus.cc:374] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:26.074126 24794 raft_consensus.cc:348] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.074460 24792 raft_consensus.cc:729] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf0b13ba9ddc45ebbc23cb02f55a397c, State: Initialized, Role: FOLLOWER
I20260811 09:04:26.074642 24794 raft_consensus.cc:374] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:26.074771 24794 raft_consensus.cc:729] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc22b5b42f17469f96afa45eba363825, State: Initialized, Role: FOLLOWER
I20260811 09:04:26.075258 24792 consensus_queue.cc:260] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.075541 24794 consensus_queue.cc:260] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.076486 24762 heartbeater.cc:503] Master 127.9.40.126:46647 was elected leader, sending a full tablet report...
I20260811 09:04:26.079535 24792 ts_tablet_manager.cc:1428] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
I20260811 09:04:26.081166 24794 ts_tablet_manager.cc:1428] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
W20260811 09:04:26.250697 24763 tablet.cc:2369] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:04:26.260344 24605 tablet.cc:2369] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:04:26.294193 24435 tablet.cc:2369] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:26.369427 24805 raft_consensus.cc:482] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:26.369802 24805 raft_consensus.cc:504] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.371687 24805 leader_election.cc:288] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf0b13ba9ddc45ebbc23cb02f55a397c (127.9.40.67:35417), 0fe5728065a04ea38e2e2390f50b533a (127.9.40.65:42283)
I20260811 09:04:26.379765 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.372229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:47775 (local address 127.9.40.65:42283)
0811 09:04:26.372505 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.372515 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.373476 (+ 961us) server_negotiation.cc:407] Connection header received
0811 09:04:26.373711 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.373716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.373807 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.373904 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:26.374587 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.375625 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.377581 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.378690 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.378816 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.378924 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.379346 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.379493 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:04:26.379801 24806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.372200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42283 (local address 127.9.40.66:47775)
0811 09:04:26.373384 (+ 1184us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:26.373403 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:26.373569 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:26.373982 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:26.373988 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:26.374017 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:26.374406 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:26.374418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.375763 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.375769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:26.377403 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:26.377421 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.378980 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.378987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:26.379151 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.379187 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:04:26.379291 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 09:04:26.379423 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1067,"thread_start_us":503,"threads_started":1}
I20260811 09:04:26.380507 24778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.372726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43203 (local address 127.9.40.67:35417)
0811 09:04:26.372903 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.372910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.373203 (+ 293us) server_negotiation.cc:407] Connection header received
0811 09:04:26.373513 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.373519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.373601 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.373719 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:26.374326 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.375429 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.378235 (+ 2806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.379343 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.379462 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.379569 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.380140 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.380269 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:04:26.380594 24807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.372127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35417 (local address 127.9.40.66:43203)
0811 09:04:26.373065 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:26.373093 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:26.373332 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:26.373733 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:26.373738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:26.373760 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:26.374178 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:26.374189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.375575 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.375580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:26.378064 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:26.378079 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.379732 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.379738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:26.379965 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.379996 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:04:26.380089 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:04:26.380203 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"mutex_wait_us":64,"thread_start_us":956,"threads_started":1}
I20260811 09:04:26.381405 24387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f907eff86a43948c48216a5accee18" candidate_uuid: "cc22b5b42f17469f96afa45eba363825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe5728065a04ea38e2e2390f50b533a" is_pre_election: true
I20260811 09:04:26.382032 24387 raft_consensus.cc:2454] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc22b5b42f17469f96afa45eba363825 in term 0.
I20260811 09:04:26.382513 24711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f907eff86a43948c48216a5accee18" candidate_uuid: "cc22b5b42f17469f96afa45eba363825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" is_pre_election: true
I20260811 09:04:26.382773 24477 leader_election.cc:302] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fe5728065a04ea38e2e2390f50b533a, cc22b5b42f17469f96afa45eba363825; no voters:
I20260811 09:04:26.382975 24711 raft_consensus.cc:2454] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc22b5b42f17469f96afa45eba363825 in term 0.
I20260811 09:04:26.383227 24805 raft_consensus.cc:2781] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:04:26.383383 24805 raft_consensus.cc:482] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:04:26.383524 24805 raft_consensus.cc:3037] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:26.385847 24805 raft_consensus.cc:504] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.386416 24805 leader_election.cc:288] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [CANDIDATE]: Term 1 election: Requested vote from peers cf0b13ba9ddc45ebbc23cb02f55a397c (127.9.40.67:35417), 0fe5728065a04ea38e2e2390f50b533a (127.9.40.65:42283)
I20260811 09:04:26.386888 24711 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f907eff86a43948c48216a5accee18" candidate_uuid: "cc22b5b42f17469f96afa45eba363825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c"
I20260811 09:04:26.386945 24387 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d6f907eff86a43948c48216a5accee18" candidate_uuid: "cc22b5b42f17469f96afa45eba363825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fe5728065a04ea38e2e2390f50b533a"
I20260811 09:04:26.387090 24711 raft_consensus.cc:3037] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:26.387154 24387 raft_consensus.cc:3037] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:26.389365 24711 raft_consensus.cc:2454] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc22b5b42f17469f96afa45eba363825 in term 1.
I20260811 09:04:26.389815 24478 leader_election.cc:302] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc22b5b42f17469f96afa45eba363825, cf0b13ba9ddc45ebbc23cb02f55a397c; no voters:
I20260811 09:04:26.390101 24805 raft_consensus.cc:2781] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:26.390421 24387 raft_consensus.cc:2454] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc22b5b42f17469f96afa45eba363825 in term 1.
I20260811 09:04:26.390851 24805 raft_consensus.cc:686] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 1 LEADER]: Becoming Leader. State: Replica: cc22b5b42f17469f96afa45eba363825, State: Running, Role: LEADER
I20260811 09:04:26.391443 24805 consensus_queue.cc:237] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } }
I20260811 09:04:26.396639 24209 catalog_manager.cc:5495] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc22b5b42f17469f96afa45eba363825 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "cc22b5b42f17469f96afa45eba363825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc22b5b42f17469f96afa45eba363825" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 33585 } health_report { overall_health: HEALTHY } } }
I20260811 09:04:26.435122 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.426243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.9.40.66:33585)
0811 09:04:26.426474 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.426481 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.429300 (+ 2819us) server_negotiation.cc:407] Connection header received
0811 09:04:26.429524 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.429528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.429628 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.429754 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:04:26.430259 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.430986 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.432967 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.433354 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.433447 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.434003 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:04:26.434523 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:04:26.434633 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.434767 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.434834 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:04:26.444445 24711 raft_consensus.cc:1261] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 1 FOLLOWER]: Refusing update from remote peer cc22b5b42f17469f96afa45eba363825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:26.444527 24387 raft_consensus.cc:1261] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 1 FOLLOWER]: Refusing update from remote peer cc22b5b42f17469f96afa45eba363825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:26.445501 24808 consensus_queue.cc:1035] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fe5728065a04ea38e2e2390f50b533a" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:26.446026 24805 consensus_queue.cc:1035] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf0b13ba9ddc45ebbc23cb02f55a397c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:26.461872 24813 mvcc.cc:205] Tried to move back new op lower bound from 7317254416146661376 to 7317254415943397376. Current Snapshot: MvccSnapshot[applied={T|T < 7317254416146661376}]
I20260811 09:04:26.483557 24816 mvcc.cc:205] Tried to move back new op lower bound from 7317254416146661376 to 7317254415943397376. Current Snapshot: MvccSnapshot[applied={T|T < 7317254416146661376}]
I20260811 09:04:26.499625 24209 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:26.499935 24209 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:26.504027 24209 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8c23f198c334591a1bf1bd5826dfeae: Sending DeleteTablet for 3 replicas of tablet d6f907eff86a43948c48216a5accee18
I20260811 09:04:26.505671 24363 tablet_service.cc:1514] Processing DeleteTablet for tablet d6f907eff86a43948c48216a5accee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40962
I20260811 09:04:26.505694 24684 tablet_service.cc:1514] Processing DeleteTablet for tablet d6f907eff86a43948c48216a5accee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58804
I20260811 09:04:26.506641 24528 tablet_service.cc:1514] Processing DeleteTablet for tablet d6f907eff86a43948c48216a5accee18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40934
I20260811 09:04:26.511672 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24257
I20260811 09:04:26.512261 24824 tablet_replica.cc:330] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: stopping tablet replica
I20260811 09:04:26.513119 24824 raft_consensus.cc:2229] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:26.513693 24824 raft_consensus.cc:2258] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:26.531910 24824 ts_tablet_manager.cc:1905] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:26.543995 24825 tablet_replica.cc:330] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: stopping tablet replica
I20260811 09:04:26.547197 24825 raft_consensus.cc:2229] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:04:26.547991 24825 raft_consensus.cc:2258] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:26.550724 24825 ts_tablet_manager.cc:1905] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:26.556811 24824 ts_tablet_manager.cc:1918] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:26.566946 24825 ts_tablet_manager.cc:1918] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:26.567600 24823 tablet_replica.cc:330] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: stopping tablet replica
I20260811 09:04:26.567879 24824 log.cc:1196] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/d6f907eff86a43948c48216a5accee18
I20260811 09:04:26.568882 24825 log.cc:1196] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/d6f907eff86a43948c48216a5accee18
I20260811 09:04:26.569562 24825 ts_tablet_manager.cc:1939] T d6f907eff86a43948c48216a5accee18 P cc22b5b42f17469f96afa45eba363825: Deleting consensus metadata
I20260811 09:04:26.571623 24190 catalog_manager.cc:4841] TS cc22b5b42f17469f96afa45eba363825 (127.9.40.66:33585): tablet d6f907eff86a43948c48216a5accee18 (table test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]) successfully deleted
I20260811 09:04:26.570371 24824 ts_tablet_manager.cc:1939] T d6f907eff86a43948c48216a5accee18 P 0fe5728065a04ea38e2e2390f50b533a: Deleting consensus metadata
I20260811 09:04:26.568974 24823 raft_consensus.cc:2229] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:26.577456 24780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.512851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.9.40.65:42283)
0811 09:04:26.513083 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.513091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.543195 (+ 30104us) server_negotiation.cc:407] Connection header received
0811 09:04:26.544437 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.544444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.544545 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.544631 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:26.548686 (+ 4055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.552101 (+ 3415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.557420 (+ 5319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.558101 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.558208 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.569162 (+ 10954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:26.569179 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:26.569201 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:26.569253 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:26.573875 (+ 4622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.574784 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.574794 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.574798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.574963 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.575387 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.575395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.575398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.575913 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:26.576227 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.576534 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.576801 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 09:04:26.578068 24823 raft_consensus.cc:2258] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:26.578534 24429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:26.579269 24190 catalog_manager.cc:4841] TS 0fe5728065a04ea38e2e2390f50b533a (127.9.40.65:42283): tablet d6f907eff86a43948c48216a5accee18 (table test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]) successfully deleted
I20260811 09:04:26.580917 24823 ts_tablet_manager.cc:1905] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:26.588171 24823 ts_tablet_manager.cc:1918] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:26.588410 24823 log.cc:1196] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/d6f907eff86a43948c48216a5accee18
I20260811 09:04:26.589013 24823 ts_tablet_manager.cc:1939] T d6f907eff86a43948c48216a5accee18 P cf0b13ba9ddc45ebbc23cb02f55a397c: Deleting consensus metadata
I20260811 09:04:26.591123 24191 catalog_manager.cc:4841] TS cf0b13ba9ddc45ebbc23cb02f55a397c (127.9.40.67:35417): tablet d6f907eff86a43948c48216a5accee18 (table test-table [id=e9e2bb5ac94f4b1ea6e8703c31af5e7a]) successfully deleted
I20260811 09:04:26.740154 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24257
I20260811 09:04:26.770538 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24439
I20260811 09:04:26.792217 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.771446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40944 (local address 127.9.40.66:33585)
0811 09:04:26.771724 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.771730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.771754 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:04:26.771952 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.771957 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.772049 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.772185 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:26.774093 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.776281 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.778177 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.779268 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.779405 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.782377 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:26.782393 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:26.782414 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:26.782502 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:26.788422 (+ 5920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.789207 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.789218 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.789227 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.789513 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.789881 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.789887 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.789891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.790565 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:26.791093 (+ 528us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.791332 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.791633 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 09:04:26.793090 24599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:26.909137 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24439
I20260811 09:04:26.934994 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24608
I20260811 09:04:26.949560 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:26.935908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58818 (local address 127.9.40.67:35417)
0811 09:04:26.936410 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:26.936415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:26.936432 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:04:26.936489 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:26.936493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:26.936572 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:26.936670 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:26.938328 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.939095 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.940528 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:26.940979 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:26.941093 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:26.943784 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:26.943797 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:26.943814 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:26.943845 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:26.946790 (+ 2945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.947585 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.947592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.947594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.947716 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:26.948079 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:26.948082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:26.948084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:26.948508 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:26.948943 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:26.949128 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 09:04:26.949296 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":290,"threads_started":1}
I20260811 09:04:26.950305 24757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:27.046144 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24608
I20260811 09:04:27.065727 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24168
I20260811 09:04:27.067080 24248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:27.186554 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24168
I20260811 09:04:27.220147 9377 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:04:27.565197 9377 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:04:27.565416 9377 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:04:27.911625 9377 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:04:27.912 UTC [23056] LOG: received smart shutdown request
2026-08-11 09:04:27.915 UTC [23056] LOG: background worker "logical replication launcher" (PID 23066) exited with exit code 1
2026-08-11 09:04:27.915 UTC [23060] LOG: shutting down
2026-08-11 09:04:28.088 UTC [23056] LOG: database system is shut down
2026-08-11T09:04:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55614 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:04:28 dist-test-slave-cbsf krb5kdc[24842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:04:28 dist-test-slave-cbsf krb5kdc[24842](info): set up 2 sockets
Aug 11 09:04:28 dist-test-slave-cbsf krb5kdc[24842](info): commencing operation
krb5kdc: starting...
W20260811 09:04:29.469954 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:04:29 dist-test-slave-cbsf krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:04:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:04:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:04:29.752372 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44229
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:30.103344 24860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:30.103652 24860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:30.103853 24860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:30.111048 24860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:30.111157 24860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:30.111248 24860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:30.111367 24860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:30.111455 24860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:30.121472 24860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:30.122844 24860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:30.124179 24860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:30.136871 24866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:30.136921 24867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:30.136822 24871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:30.141989 24860 server_base.cc:1034] running on GCE node
I20260811 09:04:30.142871 24860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:30.144335 24860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:30.145594 24860 hybrid_clock.cc:648] HybridClock initialized: now 1786439070145557 us; error 55 us; skew 500 ppm
I20260811 09:04:30.145961 24860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:30.149308 24860 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:30.150936 24860 webserver.cc:460] Webserver started at http://127.9.40.126:37659/ using document root <none> and password file <none>
I20260811 09:04:30.151636 24860 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:30.151741 24860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:30.151999 24860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:30.155105 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "7dd1acfeffbd4774b12a1bec55740259"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "6d5d399b0ab0f0c0f2a06de239834987"
server_key_iv: "527154b9383c724f4902bb85c00853d2"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.155916 24860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "7dd1acfeffbd4774b12a1bec55740259"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "6d5d399b0ab0f0c0f2a06de239834987"
server_key_iv: "527154b9383c724f4902bb85c00853d2"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.161291 24860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 09:04:30.164892 24878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:30.166275 24860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:04:30.166420 24860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "7dd1acfeffbd4774b12a1bec55740259"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "6d5d399b0ab0f0c0f2a06de239834987"
server_key_iv: "527154b9383c724f4902bb85c00853d2"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.166617 24860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:30.192026 24860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:30.196473 24860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:30.196810 24860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:30.216929 24860 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44229
I20260811 09:04:30.216961 24933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44229 every 8 connection(s)
I20260811 09:04:30.218437 24860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:30.222127 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24860
I20260811 09:04:30.222419 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:04:30.222431 24934 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:30.223351 9377 external_mini_cluster.cc:1428] Setting key 477713b1209adaead88a47c813a963ad
I20260811 09:04:30.229795 24934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
I20260811 09:04:30.254562 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.233255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.9.40.126:44229)
0811 09:04:30.235192 (+ 1937us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:30.235202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:30.235269 (+ 67us) server_negotiation.cc:407] Connection header received
0811 09:04:30.236956 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:30.236962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:30.237219 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:30.237437 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:30.238815 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.240072 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.241571 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.242150 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.242352 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.246191 (+ 3839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:30.246200 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:30.246214 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:30.246249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:30.249667 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.250403 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.250415 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.250418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.250589 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.251004 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.251009 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.251013 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.251687 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:30.251917 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:30.252532 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 09:04:30.252987 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":1665,"spinlock_wait_cycles":34560,"thread_start_us":1449,"threads_started":1}
I20260811 09:04:30.254593 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.233255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44229 (local address 127.0.0.1:32862)
0811 09:04:30.234382 (+ 1127us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.234445 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:30.236492 (+ 2047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:30.237445 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:30.237452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:30.238258 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:30.238605 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.238622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.240240 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.240246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.241424 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.241437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.242393 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.242401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.242574 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.243280 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:30.243310 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:30.245935 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:30.249818 (+ 3883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.249832 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.249845 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.250255 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.250711 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.250716 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.250719 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.250884 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.251827 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:30.251841 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:30.252127 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 09:04:30.252390 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 09:04:30.253108 (+ 718us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":213,"thread_start_us":486,"threads_started":1}
I20260811 09:04:30.257869 24934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:04:30.259058 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.258520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:45526)
0811 09:04:30.258736 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.258887 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 09:04:30.259353 24882 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
I20260811 09:04:30.261152 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.260652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:45536)
0811 09:04:30.260870 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.260962 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260811 09:04:30.261387 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:38507: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260811 09:04:30.288050 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:38507
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:30.310879 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } attempt: 1
I20260811 09:04:30.312357 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.311731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48102)
0811 09:04:30.311990 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.312140 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 09:04:30.313627 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.313128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48114)
0811 09:04:30.313325 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.313427 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:04:30.313839 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:38507: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
I20260811 09:04:30.382148 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } attempt: 2
I20260811 09:04:30.383646 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.382926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48126)
0811 09:04:30.383258 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.383400 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 09:04:30.384898 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.384403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48134)
0811 09:04:30.384584 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.384733 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 09:04:30.385123 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:38507: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
I20260811 09:04:30.476419 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } attempt: 3
I20260811 09:04:30.477921 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.477250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48142)
0811 09:04:30.477543 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.477703 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:04:30.479032 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.478604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48148)
0811 09:04:30.478770 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.478858 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 09:04:30.479240 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:38507: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
I20260811 09:04:30.622578 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } attempt: 4
I20260811 09:04:30.623955 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.623320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48162)
0811 09:04:30.623675 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.623785 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260811 09:04:30.625119 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.624675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48168)
0811 09:04:30.624855 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.624966 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 09:04:30.625402 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:38507: Network error: Client connection negotiation failed: client connection to 127.9.40.125:38507: connect: Connection refused (error 111)
W20260811 09:04:30.643520 24941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:30.643836 24941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:30.644001 24941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:30.651582 24941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:30.651700 24941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:30.651791 24941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:30.651906 24941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:30.651997 24941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:30.662434 24941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:38507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:30.663832 24941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:30.665180 24941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:30.677541 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:30.677582 24948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:30.677541 24950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:30.678851 24941 server_base.cc:1034] running on GCE node
I20260811 09:04:30.679589 24941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:30.681097 24941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:30.682303 24941 hybrid_clock.cc:648] HybridClock initialized: now 1786439070682270 us; error 31 us; skew 500 ppm
I20260811 09:04:30.682621 24941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:30.685905 24941 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260811 09:04:30.687438 24941 webserver.cc:460] Webserver started at http://127.9.40.125:33455/ using document root <none> and password file <none>
I20260811 09:04:30.688210 24941 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:30.688344 24941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:30.688673 24941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:30.691728 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data/instance:
uuid: "71708a4b45594ef39638e3a5d67b782e"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "529870b37395f5f321db5d440e12391e"
server_key_iv: "5214810f3e322ff4d8887db7f2c4c9a9"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.692553 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal/instance:
uuid: "71708a4b45594ef39638e3a5d67b782e"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "529870b37395f5f321db5d440e12391e"
server_key_iv: "5214810f3e322ff4d8887db7f2c4c9a9"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.697754 24941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260811 09:04:30.701124 24958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:30.702469 24941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:04:30.702679 24941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "71708a4b45594ef39638e3a5d67b782e"
format_stamp: "Formatted at 2026-08-11 09:04:30 on dist-test-slave-cbsf"
server_key: "529870b37395f5f321db5d440e12391e"
server_key_iv: "5214810f3e322ff4d8887db7f2c4c9a9"
server_key_version: "encryptionkey@0"
I20260811 09:04:30.702869 24941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:30.730860 24941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:30.735049 24941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:30.735357 24941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:30.755755 24941 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:38507
I20260811 09:04:30.755790 25019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:38507 every 8 connection(s)
I20260811 09:04:30.757408 24941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
I20260811 09:04:30.761121 25020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:30.767271 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 24941
I20260811 09:04:30.767503 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-1/wal/instance
I20260811 09:04:30.767976 9377 external_mini_cluster.cc:1428] Setting key 78b25a9959bfdfd90bf1776e24381334
I20260811 09:04:30.767724 25020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:30.792937 25023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.770500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45426 (local address 127.9.40.126:44229)
0811 09:04:30.771073 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:30.771080 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:30.772184 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 09:04:30.775260 (+ 3076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:30.775269 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:30.775364 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:30.775481 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:30.777630 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.778650 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.780840 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.781595 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.781752 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.785508 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:30.785517 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:30.785525 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:30.785573 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:30.789457 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.790229 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.790237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.790243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.790432 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.790853 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.790860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.790865 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.791263 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:30.791424 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:30.792125 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 09:04:30.792389 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":300,"threads_started":1}
I20260811 09:04:30.793584 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.771096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44229 (local address 127.0.0.1:45426)
0811 09:04:30.771984 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.772035 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:30.774936 (+ 2901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:30.775743 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:30.775757 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:30.776554 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:30.777440 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.777470 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.778824 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.778831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.780623 (+ 1792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.780635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.781810 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.781815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.782001 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.782701 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:30.782723 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:30.785242 (+ 2519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:30.789607 (+ 4365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.789614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.789628 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.790083 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.790552 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.790555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.790557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.790735 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.791388 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:30.791393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:30.791704 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 09:04:30.791924 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 09:04:30.792345 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":345,"threads_started":1}
I20260811 09:04:30.795563 25020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:04:30.811887 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.796503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48172)
0811 09:04:30.796759 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.796780 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:30.796894 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:30.797687 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:30.797692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:30.798200 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:30.798421 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.798433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.799884 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.799891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.800873 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.800887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.801767 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.801772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.802011 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.802679 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:30.802706 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:30.805463 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:30.809085 (+ 3622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.809092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.809100 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.809482 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.809893 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.809899 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.809903 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.810070 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.810752 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:30.810759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:30.810963 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 09:04:30.811113 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:04:30.811433 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:04:30.811990 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.796549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48172 (local address 127.9.40.125:38507)
0811 09:04:30.797248 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:30.797254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:30.797280 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:04:30.797323 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:30.797328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:30.797506 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:30.797623 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:30.798589 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.799726 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.801052 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.801620 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.801759 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.805697 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:30.805710 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:30.805726 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:30.805762 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:30.808944 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.809640 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.809646 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.809649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.809769 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.810207 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.810211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.810214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.810618 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:30.810869 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:30.811309 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 09:04:30.811566 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":392,"threads_started":1}
I20260811 09:04:30.815155 25020 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:04:30.816493 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.815861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58198)
0811 09:04:30.816090 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.816244 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:04:30.816789 24962 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:30.818629 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.818156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58208)
0811 09:04:30.818352 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.818472 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 09:04:30.818908 25020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260811 09:04:30.838331 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:39577
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39641
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:30.868309 25020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 1
I20260811 09:04:30.869740 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.869194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58210)
0811 09:04:30.869448 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.869571 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 09:04:30.871011 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.870534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58226)
0811 09:04:30.870710 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.870803 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 09:04:30.871336 25020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:30.924777 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } attempt: 5
Aug 11 09:04:30 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:04:30.939580 25020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 2
I20260811 09:04:30.940574 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.925708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48178 (local address 127.9.40.125:38507)
0811 09:04:30.925931 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:30.925935 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:30.925959 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:04:30.926224 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:30.926229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:30.926324 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:30.926433 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:30.927435 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.928316 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.930124 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.930609 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.930722 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.934844 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:30.934849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:30.934853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:30.934891 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:30.937672 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.938501 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.938506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.938508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.938636 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:30.939036 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:30.939039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:30.939041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:30.939335 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:30.939447 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:30.939989 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 09:04:30.940246 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:04:30.940572 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.925636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48178)
0811 09:04:30.925883 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.925906 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:30.926043 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:30.926494 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:30.926499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:30.927024 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:30.927260 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.927272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.928462 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.928467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.929942 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:30.929961 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:30.930774 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:30.930780 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:30.930984 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:30.931700 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:30.931725 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:30.934634 (+ 2909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:30.937842 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.937849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.937852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.938361 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.938748 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:30.938753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:30.938757 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:30.938927 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:30.939459 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:30.939464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:30.939681 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 09:04:30.939835 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 09:04:30.940161 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:04:30.940657 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.940152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58230)
0811 09:04:30.940437 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.940521 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
I20260811 09:04:30.942085 24934 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:04:30.942149 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.941682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58244)
0811 09:04:30.941877 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.941981 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 09:04:30.942435 25020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:30.943109 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.942675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58252)
0811 09:04:30.942855 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.942946 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:04:30.944103 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.943725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58254)
0811 09:04:30.943878 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.943985 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:04:30.944314 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:30.995594 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 1
I20260811 09:04:30.997129 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.996582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58266)
0811 09:04:30.996836 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.996942 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 09:04:30.998368 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:30.998050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58268)
0811 09:04:30.998185 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:30.998266 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 09:04:30.998673 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:31.033677 25020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 3
I20260811 09:04:31.034965 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.034379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58280)
0811 09:04:31.034637 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.034752 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 09:04:31.036084 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.035621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58286)
0811 09:04:31.035860 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.035953 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 09:04:31.036309 25020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:31.066952 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 2
I20260811 09:04:31.068293 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.067725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58296)
0811 09:04:31.067997 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.068105 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:04:31.069603 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.069143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58298)
0811 09:04:31.069359 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.069451 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 09:04:31.069921 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:31.176213 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 3
I20260811 09:04:31.177573 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.176987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58314)
0811 09:04:31.177245 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.177358 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 09:04:31.178748 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.178314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58322)
0811 09:04:31.178500 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.178603 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:04:31.178989 24934 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
I20260811 09:04:31.179522 25020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 4
I20260811 09:04:31.180598 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.180092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58328)
0811 09:04:31.180314 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.180415 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:04:31.181661 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.181274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58330)
0811 09:04:31.181443 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.181524 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:04:31.181906 25020 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39577: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39577: connect: Connection refused (error 111)
W20260811 09:04:31.198529 25030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:31.198809 25030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:31.198987 25030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:31.206429 25030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:31.206532 25030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:31.206635 25030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:31.206729 25030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:31.206809 25030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:31.217890 25030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39641
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44229,127.9.40.125:38507,127.9.40.124:39577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:39577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:31.219398 25030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:31.221107 25030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:31.234741 25035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:31.234665 25036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:31.234715 25039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:31.235647 25030 server_base.cc:1034] running on GCE node
I20260811 09:04:31.236476 25030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:31.238018 25030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:31.239259 25030 hybrid_clock.cc:648] HybridClock initialized: now 1786439071239204 us; error 41 us; skew 500 ppm
I20260811 09:04:31.239670 25030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439071, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:31.243141 25030 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260811 09:04:31.244660 25030 webserver.cc:460] Webserver started at http://127.9.40.124:35455/ using document root <none> and password file <none>
I20260811 09:04:31.245425 25030 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:31.245569 25030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:31.245900 25030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:31.249094 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data/instance:
uuid: "22075e6154574ed2a034af7ddb44c401"
format_stamp: "Formatted at 2026-08-11 09:04:31 on dist-test-slave-cbsf"
server_key: "0d9b1362288a2eed417331851d404193"
server_key_iv: "d890685ff984f489ea2bcfe4a8185ce6"
server_key_version: "encryptionkey@0"
I20260811 09:04:31.249944 25030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal/instance:
uuid: "22075e6154574ed2a034af7ddb44c401"
format_stamp: "Formatted at 2026-08-11 09:04:31 on dist-test-slave-cbsf"
server_key: "0d9b1362288a2eed417331851d404193"
server_key_iv: "d890685ff984f489ea2bcfe4a8185ce6"
server_key_version: "encryptionkey@0"
I20260811 09:04:31.255259 25030 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:04:31.258817 25046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:31.260130 25030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:04:31.260329 25030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "22075e6154574ed2a034af7ddb44c401"
format_stamp: "Formatted at 2026-08-11 09:04:31 on dist-test-slave-cbsf"
server_key: "0d9b1362288a2eed417331851d404193"
server_key_iv: "d890685ff984f489ea2bcfe4a8185ce6"
server_key_version: "encryptionkey@0"
I20260811 09:04:31.260540 25030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:31.290675 25030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:31.295316 25030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:31.295631 25030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:31.315424 25030 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:39577
I20260811 09:04:31.315456 25104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:39577 every 8 connection(s)
I20260811 09:04:31.317054 25030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
I20260811 09:04:31.318149 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25030
I20260811 09:04:31.318356 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786438852478101-9377-0/minicluster-data/master-2/wal/instance
I20260811 09:04:31.318887 9377 external_mini_cluster.cc:1428] Setting key 27b1394802a004c76b591baf376a6bb9
I20260811 09:04:31.321486 25105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:31.330775 25105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439071, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:31.344570 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.321001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58342 (local address 127.9.40.124:39577)
0811 09:04:31.321905 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.321922 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.321972 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:04:31.324317 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.324347 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.324878 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.325334 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.326840 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.328967 (+ 2127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.330324 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.331388 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.331582 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.336584 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.336592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.336602 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.336636 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.340263 (+ 3627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.341078 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.341086 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.341094 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.341295 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.341668 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.341673 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.341677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.342411 (+ 734us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.342618 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.343205 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.343584 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":381,"threads_started":1}
I20260811 09:04:31.350028 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.333656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44229 (local address 127.0.0.1:45436)
0811 09:04:31.334464 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.334515 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:31.334665 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:31.335336 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:31.335343 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:31.336001 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:31.336284 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.336293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.337514 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.337521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.339343 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.339359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.340094 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.340104 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.340284 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.340947 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:31.340966 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:31.343743 (+ 2777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:31.346971 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.346984 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.346991 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.347625 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.348101 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.348105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.348108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.348253 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.348922 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:31.348929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:31.349107 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 09:04:31.349283 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 09:04:31.349594 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":425,"threads_started":1}
I20260811 09:04:31.350049 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.333711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45436 (local address 127.9.40.126:44229)
0811 09:04:31.334999 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.335003 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.335018 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:04:31.335077 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.335088 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.335144 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.335271 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.336445 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.337310 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.339497 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.339894 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.339985 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.343933 (+ 3948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.343938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.343941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.343974 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.346848 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.347751 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.347756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.347760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.347959 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.348354 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.348358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.348361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.348788 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.348949 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.349389 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.349592 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1192,"thread_start_us":1098,"threads_started":1}
I20260811 09:04:31.351459 25105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:04:31.356206 24934 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 4
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439071, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:04:31.366711 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.352212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:38507 (local address 127.0.0.1:48184)
0811 09:04:31.352393 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.352412 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:31.352510 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:31.352865 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:31.352870 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:31.353335 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:31.353551 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.353566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.354860 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.354864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.355741 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.355753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.357075 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.357081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.357312 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.357891 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:31.357908 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:31.360401 (+ 2493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:31.364180 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.364184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.364186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.364601 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.365008 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.365013 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.365017 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.365166 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.365701 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:31.365706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:31.365886 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 09:04:31.366055 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 09:04:31.366338 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:04:31.366739 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.352247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48184 (local address 127.9.40.125:38507)
0811 09:04:31.352438 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.352443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.352468 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:04:31.352630 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.352633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.352731 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.352826 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.353702 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.354705 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.355895 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.356665 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.356831 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.360603 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.360608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.360611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.360643 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.364016 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.364731 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.364748 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.364751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.364886 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.365279 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.365285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.365288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.365592 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.365733 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.366165 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.366367 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:04:31.367053 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.347512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.9.40.126:44229)
0811 09:04:31.348244 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.348248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.348266 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:04:31.348325 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.348330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.348401 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.348511 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.349890 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.351157 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.352767 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.353470 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.353633 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.358448 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.358454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.358457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.358488 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.362114 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.362758 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.362763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.362766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.362897 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.363341 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.363347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.363351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.363770 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.364087 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.366418 (+ 2331us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.366659 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":399,"threads_started":1}
I20260811 09:04:31.367980 25105 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:04:31.373927 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.356805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58356)
0811 09:04:31.356989 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.357010 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:31.357116 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:31.358036 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:31.358041 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:31.358552 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:31.358776 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.358788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.360184 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.360191 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.361837 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.361856 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.363143 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.363149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.363348 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.364013 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:31.364030 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:31.366542 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:31.371224 (+ 4682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.371232 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.371237 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.371616 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.372003 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.372008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.372012 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.372224 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.372972 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:31.372978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:31.373149 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 09:04:31.373274 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 09:04:31.373544 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:04:31.374826 25109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.357529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.9.40.124:39577)
0811 09:04:31.357724 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.357731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.357751 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:04:31.357801 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.357805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.357876 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.357997 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.358944 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.360070 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.362010 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.362991 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.363126 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.366762 (+ 3636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.366771 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.366775 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.366810 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.371055 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.371753 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.371757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.371759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.371876 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.372391 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.372399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.372403 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.372811 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.374196 (+ 1385us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.374304 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.374498 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439071, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:04:31.381021 24934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: Bootstrap starting.
I20260811 09:04:31.385118 24934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:31.385458 25110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.368639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58358)
0811 09:04:31.369299 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.369316 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:31.369423 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:31.370468 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:31.370472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:31.370979 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:31.371118 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.371127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.373402 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.373410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.374638 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.374652 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.375762 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.375769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.375995 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.376549 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:31.376566 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:31.378530 (+ 1964us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:31.382744 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.382750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.382754 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.383294 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.383724 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.383728 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.383732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.383893 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.384526 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:31.384531 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:31.384751 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 09:04:31.384909 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 09:04:31.385144 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564}
I20260811 09:04:31.386260 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.369285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.9.40.124:39577)
0811 09:04:31.370225 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.370230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.370248 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:04:31.370282 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.370285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.370336 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.370447 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.372242 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.373281 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.374861 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.375597 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.375748 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.378733 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.378737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.378741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.378767 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.382345 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.383431 (+ 1086us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.383437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.383441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.383585 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.384025 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.384032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.384037 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.384376 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.384551 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.385779 (+ 1228us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.385973 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":524,"threads_started":1}
I20260811 09:04:31.386803 24934 log.cc:824] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:31.387815 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.369041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.9.40.125:38507)
0811 09:04:31.369219 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.369225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.369242 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:04:31.369290 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.369297 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.369367 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.369465 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.371365 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.372062 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.373881 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.374545 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.374671 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.380758 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.380764 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.380767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.380820 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.383872 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.384602 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.384607 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.384611 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.384799 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.385310 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.385314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.385317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.385728 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.386019 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.387224 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.387522 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:04:31.390252 24934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: No bootstrap required, opened a new log
I20260811 09:04:31.391564 25105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: Bootstrap starting.
I20260811 09:04:31.393355 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:04:31.395421 25105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:31.396801 25105 log.cc:824] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:31.398856 24934 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.399189 24934 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:31.399320 24934 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd1acfeffbd4774b12a1bec55740259, State: Initialized, Role: FOLLOWER
I20260811 09:04:31.400152 24934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.400476 25105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: No bootstrap required, opened a new log
I20260811 09:04:31.407199 25123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.407667 25123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.409868 24934 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:31.414675 25105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.415196 25105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:31.415354 25105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22075e6154574ed2a034af7ddb44c401, State: Initialized, Role: FOLLOWER
I20260811 09:04:31.418054 25105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.425503 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.425983 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.427366 25105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:04:31.449777 24884 tablet.cc:2369] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:31.456104 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.403066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45454 (local address 127.9.40.126:44229)
0811 09:04:31.403297 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.403303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.416489 (+ 13186us) server_negotiation.cc:407] Connection header received
0811 09:04:31.416887 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.416894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.416981 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.417063 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.424840 (+ 7777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.426245 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.432086 (+ 5841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.433082 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.433216 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.442144 (+ 8928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.442153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.442159 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.442206 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.445984 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.449150 (+ 3166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.449157 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.449162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.449349 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.452984 (+ 3635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.452990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.452993 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.453413 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.453689 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.455424 (+ 1735us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.455683 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:04:31.457502 25027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.402659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.9.40.125:38507)
0811 09:04:31.402853 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.402859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.417404 (+ 14545us) server_negotiation.cc:407] Connection header received
0811 09:04:31.417652 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.417657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.417738 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.417840 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.421151 (+ 3311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.422698 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.429607 (+ 6909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.430564 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.430708 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.447116 (+ 16408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.447124 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.447129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.447176 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.451117 (+ 3941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.452296 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.452304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.452307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.452475 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.454176 (+ 1701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.454181 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.454184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.454557 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.454823 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.456894 (+ 2071us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.457159 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:04:31.458319 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.417309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58362 (local address 127.9.40.124:39577)
0811 09:04:31.417569 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.417577 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.441276 (+ 23699us) server_negotiation.cc:407] Connection header received
0811 09:04:31.441485 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.441490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.441600 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.441707 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.443613 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.444811 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.446834 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.447565 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.447668 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.451223 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.451230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.451236 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.451274 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.454998 (+ 3724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.455858 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.455863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.455867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.456006 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.456504 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.456509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.456512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.456926 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.457172 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.457749 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.457969 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
W20260811 09:04:31.470820 25145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:04:31.470975 25145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:04:31.481112 25020 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } attempt: 5
I20260811 09:04:31.468678 24860 master_runner.cc:186] Error getting master registration for 127.9.40.125:38507: OK, instance_id { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" instance_seqno: 1786439070749757 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:04:31.484843 24860 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 09:04:31.483738 25159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
Aug 11 09:04:31 dist-test-slave-cbsf krb5kdc[24842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439070, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:04:31.502602 25024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.481662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39577 (local address 127.0.0.1:58368)
0811 09:04:31.481885 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:31.481906 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:31.482030 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:31.482534 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:31.482541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:31.483125 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:31.483370 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.483383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.484588 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.484594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.486003 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:31.486020 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.487859 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.487866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:31.488106 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.489994 (+ 1888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:31.490019 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:31.492985 (+ 2966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:31.499414 (+ 6429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.499419 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.499424 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.499974 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.500495 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:31.500500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:31.500504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:31.500675 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:31.501544 (+ 869us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:31.501550 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:31.501724 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 09:04:31.501876 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 09:04:31.502188 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 09:04:31.503870 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:31.481701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58368 (local address 127.9.40.124:39577)
0811 09:04:31.481863 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:31.481869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:31.481960 (+ 91us) server_negotiation.cc:407] Connection header received
0811 09:04:31.482150 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:31.482157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:31.482252 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:31.482337 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:31.483561 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.484440 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.486561 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:31.487264 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:31.487402 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:31.493254 (+ 5852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:31.493261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:31.493266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:31.493304 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:31.499239 (+ 5935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.500135 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.500141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.500145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.500328 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:31.500841 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:31.500846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:31.500850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:31.501252 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:31.501474 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:31.503159 (+ 1685us) server_negotiation.cc:299] Negotiation successful
0811 09:04:31.503410 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 09:04:31.504359 25159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:04:31.492542 25030 master_runner.cc:186] Error getting master registration for 127.9.40.125:38507: OK, instance_id { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" instance_seqno: 1786439070749757 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:04:31.507205 25030 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:04:31.511277 25020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: Bootstrap starting.
I20260811 09:04:31.514453 25020 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:31.515478 25020 log.cc:824] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:31.518165 25020 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: No bootstrap required, opened a new log
I20260811 09:04:31.526961 25020 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.527297 25020 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:31.527431 25020 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71708a4b45594ef39638e3a5d67b782e, State: Initialized, Role: FOLLOWER
I20260811 09:04:31.528015 25020 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.528954 25163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.529402 25163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.530432 25020 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:04:31.548229 25053 tablet.cc:2369] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:31.551234 24941 master_runner.cc:263] No leader master found from master 71708a4b45594ef39638e3a5d67b782e
I20260811 09:04:31.551388 24941 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 09:04:31.551618 25179 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:04:31.551777 25179 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:04:31.559840 25123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:31.560141 25123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.561830 25123 leader_election.cc:288] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71708a4b45594ef39638e3a5d67b782e (127.9.40.125:38507), 22075e6154574ed2a034af7ddb44c401 (127.9.40.124:39577)
I20260811 09:04:31.563690 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dd1acfeffbd4774b12a1bec55740259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22075e6154574ed2a034af7ddb44c401" is_pre_election: true
I20260811 09:04:31.564381 25076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd1acfeffbd4774b12a1bec55740259 in term 0.
I20260811 09:04:31.565961 24882 leader_election.cc:302] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22075e6154574ed2a034af7ddb44c401, 7dd1acfeffbd4774b12a1bec55740259; no voters:
I20260811 09:04:31.566442 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dd1acfeffbd4774b12a1bec55740259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71708a4b45594ef39638e3a5d67b782e" is_pre_election: true
I20260811 09:04:31.567173 24987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd1acfeffbd4774b12a1bec55740259 in term 0.
I20260811 09:04:31.568470 25123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:04:31.568626 25123 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:04:31.568737 25123 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:31.572441 25123 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.573664 24987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dd1acfeffbd4774b12a1bec55740259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71708a4b45594ef39638e3a5d67b782e"
I20260811 09:04:31.573885 24987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:31.577610 24987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd1acfeffbd4774b12a1bec55740259 in term 1.
I20260811 09:04:31.578171 24882 leader_election.cc:302] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71708a4b45594ef39638e3a5d67b782e, 7dd1acfeffbd4774b12a1bec55740259; no voters:
I20260811 09:04:31.580665 25123 leader_election.cc:288] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [CANDIDATE]: Term 1 election: Requested vote from peers 71708a4b45594ef39638e3a5d67b782e (127.9.40.125:38507), 22075e6154574ed2a034af7ddb44c401 (127.9.40.124:39577)
I20260811 09:04:31.581248 25123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:31.581658 25123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [term 1 LEADER]: Becoming Leader. State: Replica: 7dd1acfeffbd4774b12a1bec55740259, State: Running, Role: LEADER
I20260811 09:04:31.582708 25123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } }
I20260811 09:04:31.597693 25076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7dd1acfeffbd4774b12a1bec55740259" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22075e6154574ed2a034af7ddb44c401"
I20260811 09:04:31.597940 25076 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:31.601795 25076 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd1acfeffbd4774b12a1bec55740259 in term 1.
I20260811 09:04:31.607138 25182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd1acfeffbd4774b12a1bec55740259. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.607653 25182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:31.614667 25184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:31.617058 25184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:31.636809 25076 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [term 1 FOLLOWER]: Refusing update from remote peer 7dd1acfeffbd4774b12a1bec55740259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:31.637956 25182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:31.648839 24987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [term 1 FOLLOWER]: Refusing update from remote peer 7dd1acfeffbd4774b12a1bec55740259: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:31.649868 25182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:31.684001 25163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd1acfeffbd4774b12a1bec55740259. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.684360 25163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.687583 25185 mvcc.cc:205] Tried to move back new op lower bound from 7317254437411966976 to 7317254437229903872. Current Snapshot: MvccSnapshot[applied={T|T < 7317254437411966976}]
I20260811 09:04:31.691205 25187 mvcc.cc:205] Tried to move back new op lower bound from 7317254437411966976 to 7317254437229903872. Current Snapshot: MvccSnapshot[applied={T|T < 7317254437411966976}]
I20260811 09:04:31.695847 25123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.696244 25123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:31.698354 25184 catalog_manager.cc:1347] Generated new cluster ID: 33160b2bf5a041b9ad66fd5d8019036e
I20260811 09:04:31.698467 25184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:31.707334 25163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.707664 25163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.719208 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dd1acfeffbd4774b12a1bec55740259. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.719539 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.730707 25182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.731118 25182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:31.737154 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7dd1acfeffbd4774b12a1bec55740259" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7dd1acfeffbd4774b12a1bec55740259" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44229 } } peers { permanent_uuid: "71708a4b45594ef39638e3a5d67b782e" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 38507 } } peers { permanent_uuid: "22075e6154574ed2a034af7ddb44c401" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39577 } } }
I20260811 09:04:31.737443 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:04:31.746191 25184 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:04:31.751281 25184 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:31.772017 25184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dd1acfeffbd4774b12a1bec55740259: Generated new TSK 0
I20260811 09:04:31.773307 25184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:04:32.506743 25159 catalog_manager.cc:1259] Loaded cluster ID: 33160b2bf5a041b9ad66fd5d8019036e
I20260811 09:04:32.506846 25159 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: loading cluster ID for follower catalog manager: success
I20260811 09:04:32.509483 25159 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: acquiring CA information for follower catalog manager: success
I20260811 09:04:32.511528 25159 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22075e6154574ed2a034af7ddb44c401: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:04:32.553941 25179 catalog_manager.cc:1259] Loaded cluster ID: 33160b2bf5a041b9ad66fd5d8019036e
I20260811 09:04:32.554059 25179 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: loading cluster ID for follower catalog manager: success
I20260811 09:04:32.556389 25179 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: acquiring CA information for follower catalog manager: success
I20260811 09:04:32.558200 25179 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 71708a4b45594ef39638e3a5d67b782e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:04:32.919405 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24860
I20260811 09:04:32.921336 24927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:33.046794 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24860
I20260811 09:04:33.094067 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 24941
I20260811 09:04:33.095342 25013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:33.208048 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 24941
I20260811 09:04:33.236495 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25030
I20260811 09:04:33.237686 25099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:33.350936 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25030
2026-08-11T09:04:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5123 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:04:33 dist-test-slave-cbsf krb5kdc[25203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:04:33 dist-test-slave-cbsf krb5kdc[25203](info): set up 2 sockets
Aug 11 09:04:33 dist-test-slave-cbsf krb5kdc[25203](info): commencing operation
krb5kdc: starting...
W20260811 09:04:34.560570 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:04:34 dist-test-slave-cbsf krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:04:34.869884 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34457
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:34457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:35.230329 25220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:35.230682 25220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:35.230849 25220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:35.238266 25220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:35.238417 25220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:35.238533 25220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:35.238637 25220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:35.238727 25220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:35.248924 25220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:34457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:35.250470 25220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:35.251849 25220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:35.264652 25226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:35.264729 25225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:35.264621 25228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:35.265548 25220 server_base.cc:1034] running on GCE node
I20260811 09:04:35.266417 25220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:35.267902 25220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:35.269191 25220 hybrid_clock.cc:648] HybridClock initialized: now 1786439075269144 us; error 66 us; skew 500 ppm
I20260811 09:04:35.269599 25220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:35 dist-test-slave-cbsf krb5kdc[25203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439075, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:35.272838 25220 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:35.274497 25220 webserver.cc:460] Webserver started at http://127.9.40.126:36163/ using document root <none> and password file <none>
I20260811 09:04:35.275205 25220 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:35.275359 25220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:35.275693 25220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:35.278836 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "4fedf35ef3f9489db6741134d7f09fa3"
format_stamp: "Formatted at 2026-08-11 09:04:35 on dist-test-slave-cbsf"
server_key: "6b96d7aa4d4d50ca10c75c1b48afc52b"
server_key_iv: "c29c7fc481e1cc0129d88ad239921ec5"
server_key_version: "encryptionkey@0"
I20260811 09:04:35.279615 25220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "4fedf35ef3f9489db6741134d7f09fa3"
format_stamp: "Formatted at 2026-08-11 09:04:35 on dist-test-slave-cbsf"
server_key: "6b96d7aa4d4d50ca10c75c1b48afc52b"
server_key_iv: "c29c7fc481e1cc0129d88ad239921ec5"
server_key_version: "encryptionkey@0"
I20260811 09:04:35.284974 25220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:04:35.288528 25235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:35.289897 25220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:04:35.290094 25220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "4fedf35ef3f9489db6741134d7f09fa3"
format_stamp: "Formatted at 2026-08-11 09:04:35 on dist-test-slave-cbsf"
server_key: "6b96d7aa4d4d50ca10c75c1b48afc52b"
server_key_iv: "c29c7fc481e1cc0129d88ad239921ec5"
server_key_version: "encryptionkey@0"
I20260811 09:04:35.290305 25220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:35.319759 25220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:35.324057 25220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:35.324381 25220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:35.345062 25220 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:34457
I20260811 09:04:35.345100 25292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:34457 every 8 connection(s)
I20260811 09:04:35.346498 25220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:35.350383 25293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:35.351671 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25220
I20260811 09:04:35.351910 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:04:35.352669 9377 external_mini_cluster.cc:1428] Setting key 41bcfd8067677ae03aed76316285ef01
I20260811 09:04:35.365367 25293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3: Bootstrap starting.
Aug 11 09:04:35 dist-test-slave-cbsf krb5kdc[25203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:35.368758 25293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:35.369992 25293 log.cc:824] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:35.373631 25293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3: No bootstrap required, opened a new log
I20260811 09:04:35.378425 25296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:35.354666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54216 (local address 127.9.40.126:34457)
0811 09:04:35.355504 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:35.355524 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:35.355606 (+ 82us) server_negotiation.cc:407] Connection header received
0811 09:04:35.357499 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:35.357530 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:35.358088 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:35.358544 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:35.360054 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:35.362130 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:35.363549 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:35.364620 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:35.364827 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:35.369568 (+ 4741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:35.369579 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:35.369598 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:35.369653 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:35.373628 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:35.374402 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:35.374411 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:35.374419 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:35.374620 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:35.375003 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:35.375008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:35.375012 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:35.375770 (+ 758us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:35.376091 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:35.376507 (+ 416us) server_negotiation.cc:299] Negotiation successful
0811 09:04:35.377034 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":331,"threads_started":1}
I20260811 09:04:35.382494 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:04:35.383702 25293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } }
I20260811 09:04:35.384070 25293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:35.384186 25293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fedf35ef3f9489db6741134d7f09fa3, State: Initialized, Role: FOLLOWER
I20260811 09:04:35.384791 25293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } }
I20260811 09:04:35.385111 25293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:35.385281 25293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:35.385484 25293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:35.387768 25293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } }
I20260811 09:04:35.388255 25293 leader_election.cc:302] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fedf35ef3f9489db6741134d7f09fa3; no voters:
I20260811 09:04:35.388981 25293 leader_election.cc:288] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:04:35.389150 25298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:35.389973 25298 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 4fedf35ef3f9489db6741134d7f09fa3, State: Running, Role: LEADER
I20260811 09:04:35.390625 25298 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } }
I20260811 09:04:35.391178 25293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:35.393752 25300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fedf35ef3f9489db6741134d7f09fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } } }
I20260811 09:04:35.394270 25300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:35.395062 25301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fedf35ef3f9489db6741134d7f09fa3. Latest consensus state: current_term: 1 leader_uuid: "4fedf35ef3f9489db6741134d7f09fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fedf35ef3f9489db6741134d7f09fa3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34457 } } }
I20260811 09:04:35.395467 25301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:35.397153 25305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:35.403220 25305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:35.412325 25305 catalog_manager.cc:1347] Generated new cluster ID: 16415d33eb1b4568b028ab6358ae815d
I20260811 09:04:35.412453 25305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:35.446622 25305 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:04:35.448650 25305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:35.460004 25305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fedf35ef3f9489db6741134d7f09fa3: Generated new TSK 0
I20260811 09:04:35.461072 25305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:04:35.496675 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25220
I20260811 09:04:35.498337 25288 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:35.599066 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25220
2026-08-11T09:04:35Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2244 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:04:35 dist-test-slave-cbsf krb5kdc[25330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:04:35 dist-test-slave-cbsf krb5kdc[25330](info): set up 2 sockets
Aug 11 09:04:35 dist-test-slave-cbsf krb5kdc[25330](info): commencing operation
krb5kdc: starting...
W20260811 09:04:36.817183 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:04:37 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:04:37.112317 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:37.456043 25343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:37.456338 25343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:37.456528 25343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:37.464021 25343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:37.464123 25343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:37.464241 25343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:37.464341 25343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:37.464437 25343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:37.474406 25343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:37.475809 25343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:37.477161 25343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:37.489573 25349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:37.489538 25352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:37.489511 25350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:37.490363 25343 server_base.cc:1034] running on GCE node
I20260811 09:04:37.491187 25343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:37.492646 25343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:37.493950 25343 hybrid_clock.cc:648] HybridClock initialized: now 1786439077493898 us; error 45 us; skew 500 ppm
I20260811 09:04:37.494292 25343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:37 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:37.501906 25343 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:37.503391 25343 webserver.cc:460] Webserver started at http://127.9.40.126:33251/ using document root <none> and password file <none>
I20260811 09:04:37.504079 25343 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:37.504181 25343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:37.504467 25343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:37.507575 25343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "6ea0ac3c05934f3ebc2022fecdeb7166"
format_stamp: "Formatted at 2026-08-11 09:04:37 on dist-test-slave-cbsf"
server_key: "0d19a9cb5838a7edd99037e136ac926c"
server_key_iv: "e7890f1fe88705621a479358b6e97e62"
server_key_version: "encryptionkey@0"
I20260811 09:04:37.508293 25343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6ea0ac3c05934f3ebc2022fecdeb7166"
format_stamp: "Formatted at 2026-08-11 09:04:37 on dist-test-slave-cbsf"
server_key: "0d19a9cb5838a7edd99037e136ac926c"
server_key_iv: "e7890f1fe88705621a479358b6e97e62"
server_key_version: "encryptionkey@0"
I20260811 09:04:37.513582 25343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:04:37.517151 25361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:37.518442 25343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:04:37.518639 25343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "6ea0ac3c05934f3ebc2022fecdeb7166"
format_stamp: "Formatted at 2026-08-11 09:04:37 on dist-test-slave-cbsf"
server_key: "0d19a9cb5838a7edd99037e136ac926c"
server_key_iv: "e7890f1fe88705621a479358b6e97e62"
server_key_version: "encryptionkey@0"
I20260811 09:04:37.518836 25343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:37.535552 25343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:37.539474 25343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:37.539762 25343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:37.558754 25343 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39877
I20260811 09:04:37.558791 25419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39877 every 8 connection(s)
I20260811 09:04:37.560344 25343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:37.561058 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25343
I20260811 09:04:37.561316 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:04:37.562199 9377 external_mini_cluster.cc:1428] Setting key 273383e172128dc7f3ba1dcb1c86b846
I20260811 09:04:37.565166 25420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:04:37 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:37.578562 25420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap starting.
I20260811 09:04:37.582055 25420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:37.583076 25420 log.cc:824] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:37.585929 25424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:37.564349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.9.40.126:39877)
0811 09:04:37.565194 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:37.565206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:37.565290 (+ 84us) server_negotiation.cc:407] Connection header received
0811 09:04:37.566896 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:37.566920 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:37.567289 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:37.567611 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:37.569577 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:37.570933 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:37.572799 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:37.573442 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:37.573579 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:37.578528 (+ 4949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:37.578536 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:37.578553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:37.578593 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:37.581768 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:37.582566 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:37.582572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:37.582578 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:37.582743 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:37.583160 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:37.583164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:37.583166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:37.583713 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:37.584072 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:37.584545 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 09:04:37.584996 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":416,"threads_started":1}
I20260811 09:04:37.585975 25420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: No bootstrap required, opened a new log
I20260811 09:04:37.593366 25420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:37.593755 25420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:37.593885 25420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Initialized, Role: FOLLOWER
I20260811 09:04:37.594429 25420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:37.594674 25420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:37.594828 25420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:37.595002 25420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:37.597527 25420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:37.598006 25420 leader_election.cc:302] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ea0ac3c05934f3ebc2022fecdeb7166; no voters:
I20260811 09:04:37.598620 25420 leader_election.cc:288] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:04:37.598754 25427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:37.599555 25427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 LEADER]: Becoming Leader. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Running, Role: LEADER
I20260811 09:04:37.600127 25427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:37.600852 25420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:37.602926 25429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:37.603327 25429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:37.603833 25430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ea0ac3c05934f3ebc2022fecdeb7166. Latest consensus state: current_term: 1 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:37.604147 25430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:37.607183 25435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:37.612572 25435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:37.623629 25435 catalog_manager.cc:1347] Generated new cluster ID: de5920351ae844bf824857b1f6d2bd0a
I20260811 09:04:37.623773 25435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:37.637969 25435 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:04:37.639376 25435 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:37.649986 25435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Generated new TSK 0
I20260811 09:04:37.651027 25435 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:04:37.726338 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:38.081360 25450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:38.081804 25450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:38.081908 25450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:38.089538 25450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:38.089648 25450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:38.089766 25450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:04:38.099721 25450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:38.101059 25450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:38.102599 25450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:38.116372 25456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:38.116611 25455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:38.116395 25458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:38.117435 25450 server_base.cc:1034] running on GCE node
I20260811 09:04:38.118034 25450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:38.118973 25450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:38.120246 25450 hybrid_clock.cc:648] HybridClock initialized: now 1786439078120188 us; error 50 us; skew 500 ppm
I20260811 09:04:38.120549 25450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:38 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:38.123958 25450 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:04:38.125481 25450 webserver.cc:460] Webserver started at http://127.9.40.65:44241/ using document root <none> and password file <none>
I20260811 09:04:38.126196 25450 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:38.126348 25450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:38.126663 25450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:38.129729 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "69f261710e3c408ca289d4f0fb994325"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "954a9a681d085d4e06848cfd3dbfc2ab"
server_key_iv: "b82d09d56363578b539c4a33f8a77f51"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.130546 25450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "69f261710e3c408ca289d4f0fb994325"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "954a9a681d085d4e06848cfd3dbfc2ab"
server_key_iv: "b82d09d56363578b539c4a33f8a77f51"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.136104 25450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 09:04:38.140012 25466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.141460 25450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:04:38.141721 25450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "69f261710e3c408ca289d4f0fb994325"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "954a9a681d085d4e06848cfd3dbfc2ab"
server_key_iv: "b82d09d56363578b539c4a33f8a77f51"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.141956 25450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:38.163615 25450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:38.168123 25450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:38.168390 25450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:38.169413 25450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:38.170810 25450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:38.170889 25450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.170976 25450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:38.171029 25450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.206147 25450 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42785
I20260811 09:04:38.206167 25592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42785 every 8 connection(s)
I20260811 09:04:38.207597 25450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:04:38.216913 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25450
I20260811 09:04:38.217178 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:04:38.218111 9377 external_mini_cluster.cc:1428] Setting key bf60b042372277642caea6d71795e881
Aug 11 09:04:38 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:38.230310 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:38.210995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:52339 (local address 127.9.40.126:39877)
0811 09:04:38.213104 (+ 2109us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:38.213109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:38.213128 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:04:38.214806 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:38.214810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:38.214898 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:38.215004 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:38.216558 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.217363 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.219502 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.219969 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.220119 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:38.224153 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:38.224159 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:38.224166 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:38.224204 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:38.227033 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:38.227861 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:38.227868 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:38.227871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:38.228000 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:38.228432 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:38.228439 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:38.228442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:38.228798 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:38.228955 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:38.229671 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 09:04:38.229913 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":1995,"thread_start_us":1882,"threads_started":1}
I20260811 09:04:38.231256 25596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:38.211510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:52339)
0811 09:04:38.212785 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:38.212833 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:38.214544 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:38.215186 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:38.215194 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:38.215723 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:38.216361 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:38.216394 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.217540 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.217547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:38.219315 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:38.219328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.220212 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.220220 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:38.220473 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:38.221253 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:38.221276 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:38.223923 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:38.227202 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:38.227211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:38.227228 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:38.227722 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:38.228116 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:38.228122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:38.228125 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:38.228317 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:38.228930 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:38.228937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:38.229296 (+ 359us) client_negotiation.cc:769] Sending connection context
0811 09:04:38.229570 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 09:04:38.230128 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":852,"thread_start_us":517,"threads_started":1}
I20260811 09:04:38.233062 25593 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:38.233421 25593 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:38.234309 25593 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:38.236605 25378 ts_manager.cc:194] Registered new tserver with Master: 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:38.238116 25378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:52339
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:04:38.286528 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:38.656188 25599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:38.656476 25599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:38.656602 25599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:38.663982 25599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:38.664114 25599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:38.664227 25599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:04:38.674397 25599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:38.675822 25599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:38.677290 25599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:38.691334 25604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:38.691488 25605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:38.691635 25607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:38.692420 25599 server_base.cc:1034] running on GCE node
I20260811 09:04:38.693130 25599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:38.694159 25599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:38.695405 25599 hybrid_clock.cc:648] HybridClock initialized: now 1786439078695355 us; error 41 us; skew 500 ppm
I20260811 09:04:38.695775 25599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:38 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:38.699412 25599 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:04:38.701126 25599 webserver.cc:460] Webserver started at http://127.9.40.66:37593/ using document root <none> and password file <none>
I20260811 09:04:38.701882 25599 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:38.702040 25599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:38.702364 25599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:38.705575 25599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "4cfc86a2dd7443de907d1762d474c539"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "bef9dd80739e568e0dd7cb88b58b6c67"
server_key_iv: "fe09425ec738689def57180d9a17cd90"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.706395 25599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "4cfc86a2dd7443de907d1762d474c539"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "bef9dd80739e568e0dd7cb88b58b6c67"
server_key_iv: "fe09425ec738689def57180d9a17cd90"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.712096 25599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:04:38.715816 25614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.717326 25599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:04:38.717527 25599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "4cfc86a2dd7443de907d1762d474c539"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "bef9dd80739e568e0dd7cb88b58b6c67"
server_key_iv: "fe09425ec738689def57180d9a17cd90"
server_key_version: "encryptionkey@0"
I20260811 09:04:38.717752 25599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:38.751026 25599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:38.755293 25599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:38.755626 25599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:38.756857 25599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:38.758309 25599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:38.758424 25599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.758605 25599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:38.758685 25599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:38.794174 25599 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45343
I20260811 09:04:38.794214 25740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45343 every 8 connection(s)
I20260811 09:04:38.795589 25599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:04:38.798664 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25599
I20260811 09:04:38.798866 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:04:38.799590 9377 external_mini_cluster.cc:1428] Setting key 94d3f7aa59b47ca427fde1a29fa1464d
Aug 11 09:04:38 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439078, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:38.818250 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:38.798242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:32793 (local address 127.9.40.126:39877)
0811 09:04:38.798727 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:38.798732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:38.799888 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 09:04:38.802253 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:38.802259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:38.802350 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:38.802471 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:38.803982 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.804829 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.806429 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.806865 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.807003 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:38.811052 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:38.811058 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:38.811061 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:38.811101 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:38.814246 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:38.815113 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:38.815121 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:38.815128 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:38.815309 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:38.815836 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:38.815844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:38.815851 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:38.816256 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:38.816417 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:38.817438 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 09:04:38.817740 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":269,"threads_started":1}
I20260811 09:04:38.819063 25744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:38.798849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:32793)
0811 09:04:38.799691 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:38.799749 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:38.801943 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:38.802730 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:38.802738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:38.803244 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:38.803810 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:38.803828 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.805004 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.805008 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:38.806278 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:38.806290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:38.807139 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:38.807147 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:38.807373 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:38.808100 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:38.808170 (+ 70us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:38.810802 (+ 2632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:38.814487 (+ 3685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:38.814495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:38.814511 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:38.814979 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:38.815463 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:38.815467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:38.815469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:38.815714 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:38.816428 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:38.816458 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:38.816970 (+ 512us) client_negotiation.cc:769] Sending connection context
0811 09:04:38.817299 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 09:04:38.817774 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":343,"threads_started":1}
I20260811 09:04:38.820815 25741 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:38.821211 25741 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:38.822186 25741 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:38.824005 25378 ts_manager.cc:194] Registered new tserver with Master: 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
I20260811 09:04:38.824965 25378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:32793
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:04:38.868580 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:39.228888 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:39.229229 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:39.229400 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:39.236847 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:39.237007 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:39.237221 25747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:04:39.241854 25593 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
I20260811 09:04:39.247339 25747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:39.248684 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:39.250057 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:39.263617 25752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:39.263571 25753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:39.263643 25755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:39.264340 25747 server_base.cc:1034] running on GCE node
I20260811 09:04:39.265002 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:39.265961 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:39.267271 25747 hybrid_clock.cc:648] HybridClock initialized: now 1786439079267199 us; error 55 us; skew 500 ppm
I20260811 09:04:39.267657 25747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:39 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439079, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:39.271137 25747 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:04:39.272557 25747 webserver.cc:460] Webserver started at http://127.9.40.67:37053/ using document root <none> and password file <none>
I20260811 09:04:39.273290 25747 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:39.273396 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:39.273658 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:04:39.276575 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "43b77382b0fe4ac4833ca10aeac90985"
format_stamp: "Formatted at 2026-08-11 09:04:39 on dist-test-slave-cbsf"
server_key: "e12f86f8cdf293375ba33ffd358ad288"
server_key_iv: "0b289b12a5ba9fb6d4d11a088f83b848"
server_key_version: "encryptionkey@0"
I20260811 09:04:39.277380 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "43b77382b0fe4ac4833ca10aeac90985"
format_stamp: "Formatted at 2026-08-11 09:04:39 on dist-test-slave-cbsf"
server_key: "e12f86f8cdf293375ba33ffd358ad288"
server_key_iv: "0b289b12a5ba9fb6d4d11a088f83b848"
server_key_version: "encryptionkey@0"
I20260811 09:04:39.282488 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:04:39.285905 25763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:39.287423 25747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:04:39.287590 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "43b77382b0fe4ac4833ca10aeac90985"
format_stamp: "Formatted at 2026-08-11 09:04:39 on dist-test-slave-cbsf"
server_key: "e12f86f8cdf293375ba33ffd358ad288"
server_key_iv: "0b289b12a5ba9fb6d4d11a088f83b848"
server_key_version: "encryptionkey@0"
I20260811 09:04:39.287777 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:39.305684 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:39.310767 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:39.311066 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:39.312117 25747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:39.313810 25747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:39.313899 25747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:39.314091 25747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:39.314174 25747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:39.351456 25747 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38265
I20260811 09:04:39.351475 25888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38265 every 8 connection(s)
I20260811 09:04:39.352975 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:04:39.359444 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25747
I20260811 09:04:39.359665 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:04:39.360132 9377 external_mini_cluster.cc:1428] Setting key cb05acd2e7d8b91d718915d71fa0f8a2
Aug 11 09:04:39 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439079, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:39.374012 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.355729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34279 (local address 127.9.40.126:39877)
0811 09:04:39.356236 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.356241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.357105 (+ 864us) server_negotiation.cc:407] Connection header received
0811 09:04:39.358966 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.358972 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.359096 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.359224 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:39.361028 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.361883 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.363650 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.364087 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.364229 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.367952 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:39.367964 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:39.367968 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:39.368008 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:39.370676 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.371434 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.371442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.371450 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.371582 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.372060 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.372069 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.372072 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.372352 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:39.372528 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.373351 (+ 823us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.373588 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":253,"threads_started":1}
I20260811 09:04:39.375138 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.356153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:34279)
0811 09:04:39.356926 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.356965 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.358697 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.359444 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.359452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.360128 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:39.360824 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.360846 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.362063 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.362070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.363501 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.363513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.364323 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.364330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.364571 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.365399 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:39.365430 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:39.367725 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:39.370830 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:39.370840 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:39.370858 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:39.371296 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:39.371704 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:39.371709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:39.371712 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:39.371924 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:39.372502 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:39.372511 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:39.372963 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.373247 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.373820 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":301,"threads_started":1}
I20260811 09:04:39.376547 25889 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:39.376890 25889 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:39.377630 25889 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:39.379019 25378 ts_manager.cc:194] Registered new tserver with Master: 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265)
I20260811 09:04:39.379799 25378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:34279
I20260811 09:04:39.388293 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:04:39.405840 25892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.393363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.9.40.126:39877)
0811 09:04:39.393530 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.393534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.394035 (+ 501us) server_negotiation.cc:407] Connection header received
0811 09:04:39.394183 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.394186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.394251 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.394331 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:39.395760 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.396520 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.398000 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.398394 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.398486 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.400923 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:39.400928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:39.400931 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:39.400962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:39.403601 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.404199 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.404203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.404206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.404317 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.404657 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.404660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.404666 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.405001 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:39.405190 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.405449 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.405614 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:04:39.410248 25378 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:04:39.417225 25378 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:04:39.440917 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.430086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38265 (local address 127.0.0.1:37818)
0811 09:04:39.432018 (+ 1932us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.432039 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.432165 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.433412 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.433417 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.433450 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:39.433868 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.433880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.436049 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.436055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.438261 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.438275 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.440109 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.440116 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.440292 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.440322 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.440382 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.440510 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1760,"mutex_wait_us":706,"thread_start_us":1499,"threads_started":1}
I20260811 09:04:39.441567 25903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.430086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45343 (local address 127.0.0.1:36866)
0811 09:04:39.431058 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.431106 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.431323 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.432266 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.432273 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.432308 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:39.432844 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.432863 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.434872 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.434878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.437483 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.437498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.439705 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.439711 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.439897 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.439973 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.440128 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.440322 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":69,"thread_start_us":459,"threads_started":1}
I20260811 09:04:39.441185 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.430601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.9.40.66:45343)
0811 09:04:39.431596 (+ 995us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.431606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.431637 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:04:39.431744 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.431752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.432014 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.432832 (+ 818us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:39.433083 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.434711 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.437678 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.439223 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.439350 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.439551 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.440644 (+ 1093us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.440835 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":838,"thread_start_us":686,"threads_started":1}
I20260811 09:04:39.442826 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.430226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.9.40.67:38265)
0811 09:04:39.432827 (+ 2601us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.432836 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.432867 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:04:39.432965 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.432972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.433195 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.434057 (+ 862us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:39.434321 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.435881 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.438465 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.439870 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.442004 (+ 2134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.442258 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.442326 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.442486 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2384,"thread_start_us":2213,"threads_started":1}
I20260811 09:04:39.446825 25669 tablet_service.cc:1467] Processing CreateTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 (DEFAULT_TABLE table=test-table [id=30b83abf1e334905bfedc30dd7137d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:39.448432 25669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17cddf74454f40bbb8c18883e729e9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:39.448897 25820 tablet_service.cc:1467] Processing CreateTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 (DEFAULT_TABLE table=test-table [id=30b83abf1e334905bfedc30dd7137d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:39.450379 25820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17cddf74454f40bbb8c18883e729e9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:39.453507 25906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.432683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:51352)
0811 09:04:39.441479 (+ 8796us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.441505 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.441631 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.446541 (+ 4910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.446557 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.446590 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:39.446790 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.446801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.448750 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.448756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.450701 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.450718 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.452631 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.452637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.452868 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.452920 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.453030 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.453193 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8663,"thread_start_us":12471,"threads_started":1}
I20260811 09:04:39.453660 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.433425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51352 (local address 127.9.40.65:42785)
0811 09:04:39.445845 (+ 12420us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.445853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.445884 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:04:39.446012 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.446020 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.446288 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.446526 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:39.446987 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.448574 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.450905 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.452424 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.452574 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.452848 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.453105 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.453288 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12228,"thread_start_us":12368,"threads_started":1}
I20260811 09:04:39.458089 25522 tablet_service.cc:1467] Processing CreateTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 (DEFAULT_TABLE table=test-table [id=30b83abf1e334905bfedc30dd7137d8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:39.459635 25522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17cddf74454f40bbb8c18883e729e9f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:39.465991 25912 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Bootstrap starting.
I20260811 09:04:39.467994 25911 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Bootstrap starting.
I20260811 09:04:39.470136 25912 tablet_bootstrap.cc:654] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:39.471230 25912 log.cc:824] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:39.471611 25911 tablet_bootstrap.cc:654] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:39.472843 25911 log.cc:824] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:39.474697 25912 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: No bootstrap required, opened a new log
I20260811 09:04:39.475010 25912 ts_tablet_manager.cc:1397] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:04:39.476140 25916 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Bootstrap starting.
I20260811 09:04:39.476825 25911 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: No bootstrap required, opened a new log
I20260811 09:04:39.477201 25911 ts_tablet_manager.cc:1397] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:04:39.479925 25916 tablet_bootstrap.cc:654] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:39.481297 25916 log.cc:824] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:39.485355 25916 tablet_bootstrap.cc:492] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: No bootstrap required, opened a new log
I20260811 09:04:39.485673 25916 ts_tablet_manager.cc:1397] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 09:04:39.485780 25911 raft_consensus.cc:348] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.485786 25912 raft_consensus.cc:348] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.486195 25911 raft_consensus.cc:374] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:39.486289 25912 raft_consensus.cc:374] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:39.486423 25912 raft_consensus.cc:729] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfc86a2dd7443de907d1762d474c539, State: Initialized, Role: FOLLOWER
I20260811 09:04:39.486725 25911 raft_consensus.cc:729] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b77382b0fe4ac4833ca10aeac90985, State: Initialized, Role: FOLLOWER
I20260811 09:04:39.487115 25912 consensus_queue.cc:260] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.487413 25911 consensus_queue.cc:260] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.488591 25889 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
I20260811 09:04:39.489346 25911 ts_tablet_manager.cc:1428] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 09:04:39.497903 25741 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
I20260811 09:04:39.498713 25912 ts_tablet_manager.cc:1428] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Time spent starting tablet: real 0.024s user 0.016s sys 0.008s
I20260811 09:04:39.500881 25916 raft_consensus.cc:348] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.501430 25916 raft_consensus.cc:374] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:39.501547 25916 raft_consensus.cc:729] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f261710e3c408ca289d4f0fb994325, State: Initialized, Role: FOLLOWER
I20260811 09:04:39.502218 25916 consensus_queue.cc:260] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.504077 25916 ts_tablet_manager.cc:1428] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
W20260811 09:04:39.547744 25742 tablet.cc:2369] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:04:39.604977 25891 tablet.cc:2369] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:39.675338 25918 raft_consensus.cc:482] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:39.675622 25918 raft_consensus.cc:504] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.677400 25918 leader_election.cc:288] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343), 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:39.683365 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.677714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.9.40.67:39415)
0811 09:04:39.678055 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.678081 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.678230 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.678678 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.678682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.678704 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:39.678883 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.678891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.679949 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.679955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.681325 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.681339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.682514 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.682520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.682768 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.682795 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.682935 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.683066 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260811 09:04:39.683364 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.677968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:39415 (local address 127.9.40.65:42785)
0811 09:04:39.678179 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.678185 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.678207 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:04:39.678440 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.678448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.678536 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.678645 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:39.679049 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.679790 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.681497 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.682297 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.682451 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.682554 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.682977 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.683109 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:04:39.683998 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.678012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36905 (local address 127.9.40.66:45343)
0811 09:04:39.678218 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.678225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.678945 (+ 720us) server_negotiation.cc:407] Connection header received
0811 09:04:39.679150 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.679154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.679214 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.679315 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:39.679852 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.680548 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.682219 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.683008 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.683183 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.683277 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.683643 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.683804 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:04:39.684676 25921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.677856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45343 (local address 127.9.40.67:36905)
0811 09:04:39.678811 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:39.678828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:39.678982 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:39.679329 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:39.679334 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:39.679356 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:39.679692 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.679700 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.680725 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.680733 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.682058 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:39.682070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.683246 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.683252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:39.683409 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.683465 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 09:04:39.684140 (+ 675us) client_negotiation.cc:240] Negotiation successful
0811 09:04:39.684282 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"mutex_wait_us":1,"thread_start_us":556,"threads_started":1}
I20260811 09:04:39.684661 25543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17cddf74454f40bbb8c18883e729e9f9" candidate_uuid: "43b77382b0fe4ac4833ca10aeac90985" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f261710e3c408ca289d4f0fb994325" is_pre_election: true
I20260811 09:04:39.685423 25543 raft_consensus.cc:2454] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43b77382b0fe4ac4833ca10aeac90985 in term 0.
I20260811 09:04:39.685793 25690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17cddf74454f40bbb8c18883e729e9f9" candidate_uuid: "43b77382b0fe4ac4833ca10aeac90985" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfc86a2dd7443de907d1762d474c539" is_pre_election: true
I20260811 09:04:39.686232 25765 leader_election.cc:302] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b77382b0fe4ac4833ca10aeac90985, 69f261710e3c408ca289d4f0fb994325; no voters:
I20260811 09:04:39.686380 25690 raft_consensus.cc:2454] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43b77382b0fe4ac4833ca10aeac90985 in term 0.
I20260811 09:04:39.686695 25918 raft_consensus.cc:2781] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:04:39.686856 25918 raft_consensus.cc:482] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:04:39.686985 25918 raft_consensus.cc:3037] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:39.689324 25918 raft_consensus.cc:504] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.689946 25918 leader_election.cc:288] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [CANDIDATE]: Term 1 election: Requested vote from peers 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343), 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:39.690475 25690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17cddf74454f40bbb8c18883e729e9f9" candidate_uuid: "43b77382b0fe4ac4833ca10aeac90985" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cfc86a2dd7443de907d1762d474c539"
I20260811 09:04:39.690481 25543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17cddf74454f40bbb8c18883e729e9f9" candidate_uuid: "43b77382b0fe4ac4833ca10aeac90985" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f261710e3c408ca289d4f0fb994325"
I20260811 09:04:39.690691 25543 raft_consensus.cc:3037] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:39.690689 25690 raft_consensus.cc:3037] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:39.693166 25690 raft_consensus.cc:2454] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43b77382b0fe4ac4833ca10aeac90985 in term 1.
I20260811 09:04:39.693176 25543 raft_consensus.cc:2454] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43b77382b0fe4ac4833ca10aeac90985 in term 1.
I20260811 09:04:39.693605 25767 leader_election.cc:302] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539; no voters:
I20260811 09:04:39.693899 25918 raft_consensus.cc:2781] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:39.694386 25918 raft_consensus.cc:686] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 LEADER]: Becoming Leader. State: Replica: 43b77382b0fe4ac4833ca10aeac90985, State: Running, Role: LEADER
I20260811 09:04:39.694852 25918 consensus_queue.cc:237] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:39.699865 25378 catalog_manager.cc:5495] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 reported cstate change: term changed from 0 to 1, leader changed from <none> to 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "43b77382b0fe4ac4833ca10aeac90985" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } health_report { overall_health: UNKNOWN } } }
W20260811 09:04:39.710862 25594 tablet.cc:2369] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:39.742147 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.736608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37820 (local address 127.9.40.67:38265)
0811 09:04:39.736885 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.736890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.736924 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:04:39.737017 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.737021 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.737111 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.737264 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:04:39.737617 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.738397 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.739836 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.740240 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.740336 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.740911 (+ 575us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:04:39.741472 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:04:39.741593 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.741750 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.741868 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 09:04:39.750939 25690 raft_consensus.cc:1261] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Refusing update from remote peer 43b77382b0fe4ac4833ca10aeac90985: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:39.750949 25543 raft_consensus.cc:1261] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Refusing update from remote peer 43b77382b0fe4ac4833ca10aeac90985: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:39.752156 25918 consensus_queue.cc:1035] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:39.752521 25922 consensus_queue.cc:1035] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:39.766489 25928 mvcc.cc:205] Tried to move back new op lower bound from 7317254470648078336 to 7317254470434156544. Current Snapshot: MvccSnapshot[applied={T|T < 7317254470648078336}]
I20260811 09:04:39.782317 25377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:39.782636 25377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:39.786739 25377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Sending DeleteTablet for 3 replicas of tablet 17cddf74454f40bbb8c18883e729e9f9
I20260811 09:04:39.788028 25820 tablet_service.cc:1514] Processing DeleteTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37818
I20260811 09:04:39.788023 25522 tablet_service.cc:1514] Processing DeleteTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51352
I20260811 09:04:39.788116 25669 tablet_service.cc:1514] Processing DeleteTablet for tablet 17cddf74454f40bbb8c18883e729e9f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:36866
I20260811 09:04:39.792029 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25450
I20260811 09:04:39.805313 25940 tablet_replica.cc:330] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: stopping tablet replica
I20260811 09:04:39.806072 25939 tablet_replica.cc:330] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: stopping tablet replica
I20260811 09:04:39.806351 25941 tablet_replica.cc:330] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: stopping tablet replica
I20260811 09:04:39.806735 25939 raft_consensus.cc:2229] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:04:39.806816 25941 raft_consensus.cc:2229] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:39.807277 25940 raft_consensus.cc:2229] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:39.807503 25941 raft_consensus.cc:2258] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:39.810086 25941 ts_tablet_manager.cc:1905] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:39.810696 25940 raft_consensus.cc:2258] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:39.811585 25939 raft_consensus.cc:2258] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:39.815191 25940 ts_tablet_manager.cc:1905] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:39.818007 25941 ts_tablet_manager.cc:1918] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:39.818274 25941 log.cc:1196] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/17cddf74454f40bbb8c18883e729e9f9
I20260811 09:04:39.819020 25941 ts_tablet_manager.cc:1939] T 17cddf74454f40bbb8c18883e729e9f9 P 4cfc86a2dd7443de907d1762d474c539: Deleting consensus metadata
I20260811 09:04:39.819463 25939 ts_tablet_manager.cc:1905] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:39.821269 25365 catalog_manager.cc:4841] TS 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): tablet 17cddf74454f40bbb8c18883e729e9f9 (table test-table [id=30b83abf1e334905bfedc30dd7137d8a]) successfully deleted
Aug 11 09:04:39 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:04:39.827615 25940 ts_tablet_manager.cc:1918] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:39.827872 25940 log.cc:1196] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/17cddf74454f40bbb8c18883e729e9f9
I20260811 09:04:39.828505 25940 ts_tablet_manager.cc:1939] T 17cddf74454f40bbb8c18883e729e9f9 P 69f261710e3c408ca289d4f0fb994325: Deleting consensus metadata
I20260811 09:04:39.831248 25363 catalog_manager.cc:4841] TS 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785): tablet 17cddf74454f40bbb8c18883e729e9f9 (table test-table [id=30b83abf1e334905bfedc30dd7137d8a]) successfully deleted
I20260811 09:04:39.832134 25939 ts_tablet_manager.cc:1918] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:39.832360 25939 log.cc:1196] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/17cddf74454f40bbb8c18883e729e9f9
I20260811 09:04:39.833093 25939 ts_tablet_manager.cc:1939] T 17cddf74454f40bbb8c18883e729e9f9 P 43b77382b0fe4ac4833ca10aeac90985: Deleting consensus metadata
I20260811 09:04:39.834808 25363 catalog_manager.cc:4841] TS 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): tablet 17cddf74454f40bbb8c18883e729e9f9 (table test-table [id=30b83abf1e334905bfedc30dd7137d8a]) successfully deleted
I20260811 09:04:39.836004 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.793453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.9.40.65:42785)
0811 09:04:39.793673 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.793679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.798382 (+ 4703us) server_negotiation.cc:407] Connection header received
0811 09:04:39.800116 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.800126 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.800270 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.800366 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:39.807573 (+ 7207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.808870 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.816860 (+ 7990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.817561 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.817686 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.828412 (+ 10726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:39.828431 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:39.828450 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:39.828503 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:39.832461 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.833517 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.833526 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.833530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.833708 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.834197 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.834203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.834206 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.834685 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:39.835000 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.835376 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.835621 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:04:39.836853 25588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:39.935782 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25450
I20260811 09:04:39.966037 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25599
Aug 11 09:04:39 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:04:39.984010 25905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:39.966927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.9.40.66:45343)
0811 09:04:39.967119 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:39.967124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:39.967145 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:04:39.967302 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:39.967307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:39.967410 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:39.967525 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:39.969077 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.970264 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.971619 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:39.972232 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:39.972363 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:39.977096 (+ 4733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:39.977118 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:39.977159 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:39.977206 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:39.981050 (+ 3844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.981825 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.981835 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.981841 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.981997 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:39.982348 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:39.982354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:39.982357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:39.982862 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:39.983183 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:39.983355 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 09:04:39.983634 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:04:39.984828 25735 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:40.094786 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25599
I20260811 09:04:40.118556 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25747
Aug 11 09:04:40 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439077, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:04:40.139442 25904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:40.119392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.9.40.67:38265)
0811 09:04:40.119597 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:40.119602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:40.119621 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:04:40.119828 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:40.119832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:40.119898 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:40.119992 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:40.121976 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:40.123241 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:40.125390 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:40.125854 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:40.125957 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:40.131795 (+ 5838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:40.131816 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:40.131838 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:40.131896 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:40.136200 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:40.137082 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:40.137091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:40.137094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:40.137344 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:40.137753 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:40.137758 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:40.137763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:40.138200 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:40.138568 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:40.138746 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:04:40.138970 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:04:40.140266 25883 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:40.270442 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25747
I20260811 09:04:40.296654 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25343
I20260811 09:04:40.297890 25414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:40.412884 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25343
I20260811 09:04:40.442026 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--webserver_interface=127.9.40.126
--webserver_port=33251
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39877
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:40.793309 25947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:40.793637 25947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:40.793807 25947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:40.801267 25947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:40.801389 25947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:40.801435 25947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:40.801560 25947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:40.801626 25947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:40.811734 25947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=33251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:40.813189 25947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:40.814589 25947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:40.845181 25954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:40.845268 25955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:40.845290 25957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:40.846284 25947 server_base.cc:1034] running on GCE node
I20260811 09:04:40.847193 25947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:40.848948 25947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:40.850209 25947 hybrid_clock.cc:648] HybridClock initialized: now 1786439080850157 us; error 49 us; skew 500 ppm
I20260811 09:04:40.850653 25947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:40 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439080, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:40.854846 25947 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:40.856774 25947 webserver.cc:460] Webserver started at http://127.9.40.126:33251/ using document root <none> and password file <none>
I20260811 09:04:40.857551 25947 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:40.857687 25947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:40.864221 25947 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:04:40.867501 25965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:40.869148 25947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:04:40.869299 25947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "6ea0ac3c05934f3ebc2022fecdeb7166"
format_stamp: "Formatted at 2026-08-11 09:04:37 on dist-test-slave-cbsf"
server_key: "0d19a9cb5838a7edd99037e136ac926c"
server_key_iv: "e7890f1fe88705621a479358b6e97e62"
server_key_version: "encryptionkey@0"
I20260811 09:04:40.870416 25947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:40.899247 25947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:40.903929 25947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:40.904204 25947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:40.925961 25947 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39877
I20260811 09:04:40.926023 26023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39877 every 8 connection(s)
I20260811 09:04:40.927582 25947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:40.933459 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 25947
I20260811 09:04:40.934314 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:42785
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=44241
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:40.936025 26024 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:04:40.940156 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap starting.
I20260811 09:04:40.965375 26024 log.cc:824] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:40.980420 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:04:40.981168 26024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap complete.
I20260811 09:04:40.993821 26024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:40.995711 26024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Initialized, Role: FOLLOWER
I20260811 09:04:40.996582 26024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:40.997000 26024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:40.997226 26024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:40.997529 26024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:04:41.002743 26024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:41.003379 26024 leader_election.cc:302] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ea0ac3c05934f3ebc2022fecdeb7166; no voters:
I20260811 09:04:41.004205 26024 leader_election.cc:288] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:04:41.004383 26029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:04:41.007377 26024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:41.009037 26029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 LEADER]: Becoming Leader. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Running, Role: LEADER
I20260811 09:04:41.009950 26029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:41.013248 26031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ea0ac3c05934f3ebc2022fecdeb7166. Latest consensus state: current_term: 2 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:41.013811 26031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:41.014571 26030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:41.014986 26030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:41.017815 26041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:41.028820 26041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:41.030618 26041 catalog_manager.cc:1259] Loaded cluster ID: de5920351ae844bf824857b1f6d2bd0a
I20260811 09:04:41.030810 26041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:04:41.034658 26041 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 09:04:41.055198 26041 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 09:04:41.066104 26041 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:04:41.067497 26041 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Loaded TSK: 0
I20260811 09:04:41.068832 26041 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:04:41.338300 26027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:41.338563 26027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:41.338670 26027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:41.346160 26027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:41.346323 26027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:41.346505 26027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:04:41.356990 26027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:42785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=44241
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:41.358732 26027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:41.360219 26027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:41.374713 26055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:41.374991 26054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:41.374801 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:41.376796 26027 server_base.cc:1034] running on GCE node
I20260811 09:04:41.377647 26027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:41.378939 26027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:41.380247 26027 hybrid_clock.cc:648] HybridClock initialized: now 1786439081380171 us; error 55 us; skew 500 ppm
I20260811 09:04:41.380679 26027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:41 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439081, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:41.385017 26027 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:04:41.386955 26027 webserver.cc:460] Webserver started at http://127.9.40.65:44241/ using document root <none> and password file <none>
I20260811 09:04:41.387897 26027 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:41.388067 26027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:41.395740 26027 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:04:41.399458 26066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:41.401281 26027 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:04:41.401491 26027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "69f261710e3c408ca289d4f0fb994325"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "954a9a681d085d4e06848cfd3dbfc2ab"
server_key_iv: "b82d09d56363578b539c4a33f8a77f51"
server_key_version: "encryptionkey@0"
I20260811 09:04:41.403016 26027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:41.447273 26027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:41.452301 26027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:41.452608 26027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:41.453821 26027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:41.455399 26027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:41.455494 26027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:41.455636 26027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:41.455741 26027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:41.498682 26027 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42785
I20260811 09:04:41.498796 26193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42785 every 8 connection(s)
I20260811 09:04:41.500197 26027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:04:41.507391 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26027
I20260811 09:04:41.508271 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:45343
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=37593
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:41 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439081, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:41.527263 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:41.503399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43575 (local address 127.9.40.126:39877)
0811 09:04:41.504105 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:41.504117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:41.504543 (+ 426us) server_negotiation.cc:407] Connection header received
0811 09:04:41.506717 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:41.506736 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:41.507225 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:41.507666 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:41.509686 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:41.511780 (+ 2094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:41.513540 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:41.514688 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:41.514887 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:41.518656 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:41.518669 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:41.518692 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:41.518742 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:41.522730 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:41.523701 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:41.523708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:41.523713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:41.523863 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:41.524347 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:41.524350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:41.524352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:41.524895 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:41.525062 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:41.525952 (+ 890us) server_negotiation.cc:299] Negotiation successful
0811 09:04:41.526283 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":333,"threads_started":1}
I20260811 09:04:41.527693 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:41.503483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:43575)
0811 09:04:41.504354 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:41.504409 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:41.506210 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:41.507909 (+ 1699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:41.507924 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:41.508707 (+ 783us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:41.509455 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:41.509476 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:41.511949 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:41.511953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:41.513379 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:41.513391 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:41.514930 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:41.514935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:41.515128 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:41.515859 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:41.515882 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:41.518340 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:41.522873 (+ 4533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:41.522885 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:41.522903 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:41.523541 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:41.523978 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:41.523986 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:41.523992 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:41.524231 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:41.525033 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:41.525044 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:41.525467 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 09:04:41.525773 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 09:04:41.526370 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":403,"threads_started":1}
I20260811 09:04:41.530522 26194 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:41.530855 26194 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:41.531611 26194 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:41.533823 25983 ts_manager.cc:194] Registered new tserver with Master: 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:41.535409 25983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:43575
W20260811 09:04:41.864225 26198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:41.864509 26198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:41.864658 26198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:41.872082 26198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:41.872200 26198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:41.872382 26198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:04:41.882603 26198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:45343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=37593
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:41.883945 26198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:41.885483 26198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:41.899302 26204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:41.899348 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:41.899394 26203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:41.900600 26198 server_base.cc:1034] running on GCE node
I20260811 09:04:41.901347 26198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:41.902352 26198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:41.903589 26198 hybrid_clock.cc:648] HybridClock initialized: now 1786439081903530 us; error 49 us; skew 500 ppm
I20260811 09:04:41.903913 26198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:41 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439081, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:41.907400 26198 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:04:41.909335 26198 webserver.cc:460] Webserver started at http://127.9.40.66:37593/ using document root <none> and password file <none>
I20260811 09:04:41.910166 26198 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:41.910346 26198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:41.916314 26198 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 09:04:41.919349 26217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:41.920907 26198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:04:41.921131 26198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "4cfc86a2dd7443de907d1762d474c539"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "bef9dd80739e568e0dd7cb88b58b6c67"
server_key_iv: "fe09425ec738689def57180d9a17cd90"
server_key_version: "encryptionkey@0"
I20260811 09:04:41.922335 26198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:41.958256 26198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:41.962774 26198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:41.963050 26198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:41.964126 26198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:41.965584 26198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:41.965678 26198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:41.965762 26198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:41.965824 26198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:42.003837 26198 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45343
I20260811 09:04:42.003940 26340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45343 every 8 connection(s)
I20260811 09:04:42.005499 26198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:04:42.013863 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26198
I20260811 09:04:42.014609 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:38265
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=37053
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:04:42 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439081, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:42.031380 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.008388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:38669 (local address 127.9.40.126:39877)
0811 09:04:42.008604 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.008608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.009950 (+ 1342us) server_negotiation.cc:407] Connection header received
0811 09:04:42.012004 (+ 2054us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.012008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.012130 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.012243 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:42.014338 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.015654 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.018321 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.019214 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.019388 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.024390 (+ 5002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:42.024400 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:42.024407 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:42.024459 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:42.027848 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.028940 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.028945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.028948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.029071 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.029609 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.029612 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.029614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.029904 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:42.030020 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.030940 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.031139 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:04:42.033143 26343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.009018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:38669)
0811 09:04:42.009762 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.009806 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.011710 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.012458 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.012465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.013280 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:42.014130 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.014154 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.015862 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.015870 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.018142 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.018161 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.019475 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.019485 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.019827 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.020597 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:42.020634 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:42.024126 (+ 3492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:42.028028 (+ 3902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:42.028043 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:42.028065 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:42.028775 (+ 710us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:42.029202 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:42.029210 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:42.029216 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:42.029489 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:42.030043 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:42.030054 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:42.030528 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.030843 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.031500 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":321,"threads_started":1}
I20260811 09:04:42.035027 26341 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:42.035391 26341 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:42.036144 26341 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:42.037761 25983 ts_manager.cc:194] Registered new tserver with Master: 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
I20260811 09:04:42.039068 25983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:38669
W20260811 09:04:42.389883 26344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:42.390249 26344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:42.390364 26344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:42.398366 26344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:42.398495 26344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:42.398623 26344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:04:42.408912 26344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:38265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=37053
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:42.410269 26344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:42.411657 26344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:42.425961 26349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:42.425874 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:42.425837 26355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:42.427443 26344 server_base.cc:1034] running on GCE node
I20260811 09:04:42.428073 26344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:42.429155 26344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:42.430406 26344 hybrid_clock.cc:648] HybridClock initialized: now 1786439082430370 us; error 50 us; skew 500 ppm
I20260811 09:04:42.430756 26344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:42 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439082, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:42.434191 26344 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:04:42.435673 26344 webserver.cc:460] Webserver started at http://127.9.40.67:37053/ using document root <none> and password file <none>
I20260811 09:04:42.436367 26344 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:42.436493 26344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:42.442229 26344 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 09:04:42.445070 26362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:42.446463 26344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:04:42.446696 26344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "43b77382b0fe4ac4833ca10aeac90985"
format_stamp: "Formatted at 2026-08-11 09:04:39 on dist-test-slave-cbsf"
server_key: "e12f86f8cdf293375ba33ffd358ad288"
server_key_iv: "0b289b12a5ba9fb6d4d11a088f83b848"
server_key_version: "encryptionkey@0"
I20260811 09:04:42.447757 26344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:42.483422 26344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:42.487707 26344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:42.488016 26344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:42.489174 26344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:42.490696 26344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:42.490821 26344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:42.490967 26344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:42.491063 26344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:42.527675 26344 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38265
I20260811 09:04:42.527727 26494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38265 every 8 connection(s)
I20260811 09:04:42.529242 26344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:04:42.531266 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26344
I20260811 09:04:42.538709 26194 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
Aug 11 09:04:42 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439082, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:04:42.552205 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.532517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56048 (local address 127.9.40.126:39877)
0811 09:04:42.532935 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.532943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.533304 (+ 361us) server_negotiation.cc:407] Connection header received
0811 09:04:42.533526 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.533530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.533618 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.533738 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:42.535441 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.536423 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.537874 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.538341 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.538482 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.545793 (+ 7311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:42.545803 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:42.545811 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:42.545851 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:42.549451 (+ 3600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.550194 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.550205 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.550207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.550339 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.550750 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.550755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.550757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.551044 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:42.551315 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.551666 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.551833 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":214,"threads_started":1}
I20260811 09:04:42.553318 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.532062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60309 (local address 127.9.40.126:39877)
0811 09:04:42.532550 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.532557 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.533915 (+ 1358us) server_negotiation.cc:407] Connection header received
0811 09:04:42.535854 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.535859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.535930 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.536050 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:42.537662 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.538623 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.540642 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.541149 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.541287 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.546686 (+ 5399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:42.546692 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:42.546695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:42.546735 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:42.549595 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.550354 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.550359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.550362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.550483 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.550942 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.550947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.550950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.551306 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:42.551459 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.552682 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.552937 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":238,"threads_started":1}
I20260811 09:04:42.554291 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.532794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:60309)
0811 09:04:42.533740 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.533789 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.535607 (+ 1818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.536261 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.536269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.536876 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:42.537468 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.537493 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.538792 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.538797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.540481 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.540494 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.541397 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.541406 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.541648 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.543777 (+ 2129us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:42.543806 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:42.546423 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:42.549748 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:42.549759 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:42.549776 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:42.550216 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:42.550607 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:42.550612 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:42.550615 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:42.550815 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:42.551470 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:42.551479 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:42.551945 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.552306 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.552938 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":398,"threads_started":1}
I20260811 09:04:42.555823 26495 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39877
I20260811 09:04:42.556159 26495 heartbeater.cc:464] Registering TS with master...
I20260811 09:04:42.557029 26495 heartbeater.cc:511] Master 127.9.40.126:39877 requested a full tablet report, sending...
I20260811 09:04:42.558522 25983 ts_manager.cc:194] Registered new tserver with Master: 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265)
I20260811 09:04:42.559291 25983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60309
I20260811 09:04:42.567134 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:04:42.574153 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.568049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.9.40.126:39877)
0811 09:04:42.568236 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.568241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.568861 (+ 620us) server_negotiation.cc:407] Connection header received
0811 09:04:42.569051 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.569055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.569200 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.569333 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:04:42.569728 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.570660 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.572062 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.572458 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.572548 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.573098 (+ 550us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:04:42.573606 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:04:42.573774 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.573815 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.573881 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:04:42.575136 25983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:04:42.582494 25983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:04:42.606295 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.594925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37798 (local address 127.9.40.67:38265)
0811 09:04:42.597890 (+ 2965us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.597896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.597922 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:04:42.598004 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.598009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.598189 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.598339 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:42.599389 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.601077 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.603597 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.605068 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.605211 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.605447 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.605765 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.605930 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2800,"thread_start_us":2650,"threads_started":1}
I20260811 09:04:42.606349 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.594548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38265 (local address 127.0.0.1:37798)
0811 09:04:42.596084 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.596147 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.596356 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.598409 (+ 2053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.598421 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.598479 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:42.599101 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.599118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.601273 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.601279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.603350 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.603363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.605295 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.605302 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.605507 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.605556 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.605698 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.605884 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"mutex_wait_us":4,"thread_start_us":4139,"threads_started":1}
I20260811 09:04:42.607784 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.595865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40600 (local address 127.9.40.66:45343)
0811 09:04:42.597385 (+ 1520us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.597397 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.599064 (+ 1667us) server_negotiation.cc:407] Connection header received
0811 09:04:42.599306 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.599312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.599559 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.599767 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:42.600733 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.602272 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.604884 (+ 2612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.606314 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.606447 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.606648 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.607239 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.607414 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1291,"thread_start_us":1098,"threads_started":1}
I20260811 09:04:42.608839 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.595116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45343 (local address 127.0.0.1:40600)
0811 09:04:42.598934 (+ 3818us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.598949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.599130 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.600016 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.600021 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.600047 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:42.600494 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.600509 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.602399 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.602405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.604668 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.604683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.606840 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.606847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.607034 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.607066 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.608139 (+ 1073us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.608288 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3704,"mutex_wait_us":22,"thread_start_us":957,"threads_started":1}
I20260811 09:04:42.609462 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.599428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53786 (local address 127.9.40.65:42785)
0811 09:04:42.600037 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.600045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.600870 (+ 825us) server_negotiation.cc:407] Connection header received
0811 09:04:42.601156 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.601167 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.601382 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.601559 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:42.602327 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.604000 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.606359 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.607727 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.607844 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.608119 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.608951 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.609116 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":308,"threads_started":1}
I20260811 09:04:42.609831 26512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.600029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53786)
0811 09:04:42.600761 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.600778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.600966 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.601776 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.601779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.601797 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:42.602137 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.602146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.604168 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.604173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.606157 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.606177 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.608582 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.608586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.608756 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.608789 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.609115 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.609252 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":411,"threads_started":1}
I20260811 09:04:42.610817 26417 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 (DEFAULT_TABLE table=test-table [id=8eb33d6c7c724bfd820f07adce5625da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:42.612597 26417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd60ea29e8f48e3bbebde82e6e556d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:42.612753 26268 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 (DEFAULT_TABLE table=test-table [id=8eb33d6c7c724bfd820f07adce5625da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:42.613137 26119 tablet_service.cc:1467] Processing CreateTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 (DEFAULT_TABLE table=test-table [id=8eb33d6c7c724bfd820f07adce5625da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:04:42.614084 26268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd60ea29e8f48e3bbebde82e6e556d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:42.614305 26119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9bd60ea29e8f48e3bbebde82e6e556d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:04:42.626698 26513 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Bootstrap starting.
I20260811 09:04:42.628718 26514 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Bootstrap starting.
I20260811 09:04:42.630203 26515 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Bootstrap starting.
I20260811 09:04:42.632136 26513 tablet_bootstrap.cc:654] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:42.632138 26514 tablet_bootstrap.cc:654] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:42.633061 26515 tablet_bootstrap.cc:654] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Neither blocks nor log segments found. Creating new log.
I20260811 09:04:42.633471 26514 log.cc:824] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:42.633473 26513 log.cc:824] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:42.634169 26515 log.cc:824] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:42.636391 26513 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: No bootstrap required, opened a new log
I20260811 09:04:42.636423 26514 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: No bootstrap required, opened a new log
I20260811 09:04:42.636670 26513 ts_tablet_manager.cc:1397] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 09:04:42.636740 26514 ts_tablet_manager.cc:1397] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 09:04:42.638705 26515 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: No bootstrap required, opened a new log
I20260811 09:04:42.639034 26515 ts_tablet_manager.cc:1397] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:04:42.644903 26514 raft_consensus.cc:348] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.645313 26514 raft_consensus.cc:374] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:42.645422 26514 raft_consensus.cc:729] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43b77382b0fe4ac4833ca10aeac90985, State: Initialized, Role: FOLLOWER
I20260811 09:04:42.646063 26514 consensus_queue.cc:260] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.647251 26495 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
I20260811 09:04:42.647821 26514 ts_tablet_manager.cc:1428] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 09:04:42.649076 26513 raft_consensus.cc:348] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.649537 26513 raft_consensus.cc:374] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:42.649636 26513 raft_consensus.cc:729] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f261710e3c408ca289d4f0fb994325, State: Initialized, Role: FOLLOWER
I20260811 09:04:42.649873 26515 raft_consensus.cc:348] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.650306 26515 raft_consensus.cc:374] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:04:42.650425 26515 raft_consensus.cc:729] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cfc86a2dd7443de907d1762d474c539, State: Initialized, Role: FOLLOWER
I20260811 09:04:42.650342 26513 consensus_queue.cc:260] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.651073 26515 consensus_queue.cc:260] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.652873 26341 heartbeater.cc:503] Master 127.9.40.126:39877 was elected leader, sending a full tablet report...
I20260811 09:04:42.653268 26515 ts_tablet_manager.cc:1428] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 09:04:42.656031 26513 ts_tablet_manager.cc:1428] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Time spent starting tablet: real 0.019s user 0.005s sys 0.010s
W20260811 09:04:42.752892 26195 tablet.cc:2369] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:04:42.757558 26342 tablet.cc:2369] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:04:42.781064 26496 tablet.cc:2369] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:04:42.842962 26521 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:42.843295 26521 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.845196 26521 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:42.851814 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.845654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44955 (local address 127.9.40.67:38265)
0811 09:04:42.845857 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.845862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.846252 (+ 390us) server_negotiation.cc:407] Connection header received
0811 09:04:42.846554 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.846560 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.846643 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.846717 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:42.847419 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.848186 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.850032 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.850821 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.850981 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.851108 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.851390 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.851523 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:04:42.852108 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.845380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38265 (local address 127.9.40.66:44955)
0811 09:04:42.846124 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.846148 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.846324 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.846889 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.846892 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.846912 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:42.847267 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.847275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.848321 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.848329 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.849871 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.849883 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.851046 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.851052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.851223 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.851248 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.851360 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.851494 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"spinlock_wait_cycles":17280,"thread_start_us":377,"threads_started":1}
I20260811 09:04:42.852424 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.846413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44905 (local address 127.9.40.65:42785)
0811 09:04:42.846656 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.846662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.846919 (+ 257us) server_negotiation.cc:407] Connection header received
0811 09:04:42.847157 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.847163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.847235 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.847341 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:04:42.847878 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.848659 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.850433 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.851285 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.851443 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.851566 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.852003 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.852147 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:04:42.853219 26523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.846207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.9.40.66:44905)
0811 09:04:42.846802 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:42.846816 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:42.847000 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:42.847361 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:42.847365 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:42.847389 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:04:42.847722 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.847730 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.848853 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.848858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.850271 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:42.850283 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.851522 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.851528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:42.851731 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.851790 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 09:04:42.852630 (+ 840us) client_negotiation.cc:240] Negotiation successful
0811 09:04:42.852803 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":292,"threads_started":1}
I20260811 09:04:42.853247 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bd60ea29e8f48e3bbebde82e6e556d4" candidate_uuid: "4cfc86a2dd7443de907d1762d474c539" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43b77382b0fe4ac4833ca10aeac90985" is_pre_election: true
I20260811 09:04:42.853869 26441 raft_consensus.cc:2454] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cfc86a2dd7443de907d1762d474c539 in term 0.
I20260811 09:04:42.854207 26143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bd60ea29e8f48e3bbebde82e6e556d4" candidate_uuid: "4cfc86a2dd7443de907d1762d474c539" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f261710e3c408ca289d4f0fb994325" is_pre_election: true
I20260811 09:04:42.854624 26219 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539; no voters:
I20260811 09:04:42.854779 26143 raft_consensus.cc:2454] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4cfc86a2dd7443de907d1762d474c539 in term 0.
I20260811 09:04:42.855098 26521 raft_consensus.cc:2781] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:04:42.855252 26521 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:04:42.855391 26521 raft_consensus.cc:3037] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:42.857970 26521 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.858598 26521 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [CANDIDATE]: Term 1 election: Requested vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785)
I20260811 09:04:42.859198 26441 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bd60ea29e8f48e3bbebde82e6e556d4" candidate_uuid: "4cfc86a2dd7443de907d1762d474c539" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43b77382b0fe4ac4833ca10aeac90985"
I20260811 09:04:42.859279 26143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9bd60ea29e8f48e3bbebde82e6e556d4" candidate_uuid: "4cfc86a2dd7443de907d1762d474c539" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69f261710e3c408ca289d4f0fb994325"
I20260811 09:04:42.859417 26441 raft_consensus.cc:3037] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:42.859552 26143 raft_consensus.cc:3037] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:04:42.861876 26441 raft_consensus.cc:2454] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cfc86a2dd7443de907d1762d474c539 in term 1.
I20260811 09:04:42.861981 26143 raft_consensus.cc:2454] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4cfc86a2dd7443de907d1762d474c539 in term 1.
I20260811 09:04:42.862345 26219 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539; no voters:
I20260811 09:04:42.862694 26521 raft_consensus.cc:2781] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:04:42.863171 26521 raft_consensus.cc:686] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 1 LEADER]: Becoming Leader. State: Replica: 4cfc86a2dd7443de907d1762d474c539, State: Running, Role: LEADER
I20260811 09:04:42.863807 26521 consensus_queue.cc:237] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:42.868484 25982 catalog_manager.cc:5495] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "4cfc86a2dd7443de907d1762d474c539" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } health_report { overall_health: UNKNOWN } } }
I20260811 09:04:42.908587 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.902606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.9.40.66:45343)
0811 09:04:42.902810 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.902814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.902833 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:04:42.903010 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.903014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.903092 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.903217 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:04:42.903741 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.904483 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.906151 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.906567 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.906667 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.907230 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:04:42.907847 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:04:42.907998 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.908161 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.908230 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:04:42.917450 26441 raft_consensus.cc:1261] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Refusing update from remote peer 4cfc86a2dd7443de907d1762d474c539: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:42.917451 26143 raft_consensus.cc:1261] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Refusing update from remote peer 4cfc86a2dd7443de907d1762d474c539: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:04:42.918545 26524 consensus_queue.cc:1035] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:42.918977 26521 consensus_queue.cc:1035] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:04:42.934353 26529 mvcc.cc:205] Tried to move back new op lower bound from 7317254483619086336 to 7317254483414515712. Current Snapshot: MvccSnapshot[applied={T|T < 7317254483619086336}]
I20260811 09:04:42.935846 26527 mvcc.cc:205] Tried to move back new op lower bound from 7317254483619086336 to 7317254483414515712. Current Snapshot: MvccSnapshot[applied={T|T < 7317254483619086336}]
I20260811 09:04:42.952929 25982 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:42.953445 25982 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:04:42.957448 25982 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Sending DeleteTablet for 3 replicas of tablet 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:42.958890 26268 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40600
I20260811 09:04:42.959052 26119 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53786
I20260811 09:04:42.959221 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26027
I20260811 09:04:42.959879 26417 tablet_service.cc:1514] Processing DeleteTablet for tablet 9bd60ea29e8f48e3bbebde82e6e556d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:04:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37798
I20260811 09:04:42.963338 26538 tablet_replica.cc:330] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: stopping tablet replica
I20260811 09:04:42.963833 26538 raft_consensus.cc:2229] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:42.964381 26538 raft_consensus.cc:2258] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:42.966924 26537 tablet_replica.cc:330] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: stopping tablet replica
I20260811 09:04:42.968465 26537 raft_consensus.cc:2229] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:04:42.968333 26540 tablet_replica.cc:330] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: stopping tablet replica
I20260811 09:04:42.968963 26540 raft_consensus.cc:2229] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:04:42.969242 26537 raft_consensus.cc:2258] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:42.969480 26540 raft_consensus.cc:2258] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:04:42.971765 26540 ts_tablet_manager.cc:1905] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:42.972051 26538 ts_tablet_manager.cc:1905] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:42.976487 26537 ts_tablet_manager.cc:1905] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:04:42.979358 26540 ts_tablet_manager.cc:1918] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:42.979530 26540 log.cc:1196] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:42.980126 26540 ts_tablet_manager.cc:1939] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 43b77382b0fe4ac4833ca10aeac90985: Deleting consensus metadata
I20260811 09:04:42.980585 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:42.960324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.9.40.65:42785)
0811 09:04:42.960583 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:42.960592 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:42.960621 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:04:42.960863 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:42.960873 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:42.960983 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:42.961120 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:42.962890 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.964436 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.966402 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:42.967244 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:42.967420 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:42.971339 (+ 3919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:42.971359 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:42.971378 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:42.971423 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:42.975955 (+ 4532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.977065 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.977077 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.977083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.977273 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:42.977730 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:42.977735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:42.977738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:42.978228 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:42.978528 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:42.979850 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0811 09:04:42.980099 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:04:42.981761 26189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:42.982614 25967 catalog_manager.cc:4841] TS 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): tablet 9bd60ea29e8f48e3bbebde82e6e556d4 (table test-table [id=8eb33d6c7c724bfd820f07adce5625da]) successfully deleted
I20260811 09:04:42.984609 26537 ts_tablet_manager.cc:1918] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:04:42.984841 26537 log.cc:1196] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:42.985554 26537 ts_tablet_manager.cc:1939] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 4cfc86a2dd7443de907d1762d474c539: Deleting consensus metadata
I20260811 09:04:42.987378 25969 catalog_manager.cc:4841] TS 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): tablet 9bd60ea29e8f48e3bbebde82e6e556d4 (table test-table [id=8eb33d6c7c724bfd820f07adce5625da]) successfully deleted
I20260811 09:04:43.109403 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26027
W20260811 09:04:43.133291 25967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:42785 (error 108)
I20260811 09:04:43.133499 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26198
I20260811 09:04:43.137444 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.134851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53804)
0811 09:04:43.137066 (+ 2215us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.137228 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2076,"thread_start_us":1943,"threads_started":1}
W20260811 09:04:43.137750 25967 catalog_manager.cc:4584] TS 69f261710e3c408ca289d4f0fb994325 (127.9.40.65:42785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9bd60ea29e8f48e3bbebde82e6e556d4: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
I20260811 09:04:43.149780 26510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.134303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.9.40.66:45343)
0811 09:04:43.134500 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:43.134506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:43.134713 (+ 207us) server_negotiation.cc:407] Connection header received
0811 09:04:43.134936 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:43.134942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:43.135029 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:43.135140 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:43.137382 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:43.138381 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:43.139756 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:43.140274 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:43.140437 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:43.143282 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:43.143296 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:43.143312 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:43.143353 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:43.146709 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:43.147467 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:43.147475 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:43.147478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:43.147619 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:43.148065 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:43.148071 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:43.148078 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:43.148489 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:43.149026 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:43.149181 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 09:04:43.149405 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:04:43.150552 26336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:43.191694 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.191091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53816)
0811 09:04:43.191386 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.191518 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 09:04:43.192977 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.192489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53820)
0811 09:04:43.192673 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.192791 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:04:43.253633 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.253077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53824)
0811 09:04:43.253319 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.253440 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 09:04:43.254915 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.254496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53826)
0811 09:04:43.254667 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.254752 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:04:43.271806 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26198
I20260811 09:04:43.297438 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26344
I20260811 09:04:43.311671 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.298288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37802 (local address 127.9.40.67:38265)
0811 09:04:43.298487 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:43.298492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:43.298510 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:04:43.298647 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:43.298651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:43.298735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:43.298819 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:43.300302 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:43.301129 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:43.302474 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:43.302862 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:43.302950 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:43.305669 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:43.305684 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:43.305700 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:43.305736 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:43.308734 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:43.309592 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:43.309599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:43.309601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:43.309735 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:43.310130 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:43.310133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:43.310135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:43.310470 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:43.310753 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:43.311040 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 09:04:43.311282 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:04:43.312445 26490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:43.335700 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.335170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53830)
0811 09:04:43.335410 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.335519 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:04:43.337037 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.336545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53844)
0811 09:04:43.336791 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.336874 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 09:04:43.405165 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26344
I20260811 09:04:43.428642 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 25947
I20260811 09:04:43.430177 26018 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:04:43.547609 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:43.546937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42785 (local address 127.0.0.1:53852)
0811 09:04:43.547262 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:43.547373 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 09:04:43.547564 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 25947
I20260811 09:04:43.575217 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--webserver_interface=127.9.40.126
--webserver_port=33251
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39877
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:43.924329 26548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:43.924659 26548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:43.924847 26548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:43.932307 26548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:04:43.932420 26548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:43.932467 26548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:43.932550 26548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:04:43.932618 26548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:04:43.942922 26548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39877
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=33251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:43.944363 26548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:43.945803 26548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:43.958292 26553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:43.958271 26554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:43.959076 26556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:43.960160 26548 server_base.cc:1034] running on GCE node
I20260811 09:04:43.961040 26548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:43.962595 26548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:43.963846 26548 hybrid_clock.cc:648] HybridClock initialized: now 1786439083963796 us; error 71 us; skew 500 ppm
I20260811 09:04:43.964219 26548 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:43 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439083, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:43.967424 26548 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:04:43.969112 26548 webserver.cc:460] Webserver started at http://127.9.40.126:33251/ using document root <none> and password file <none>
I20260811 09:04:43.969772 26548 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:43.969879 26548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:43.975328 26548 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:04:43.978221 26564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:43.979992 26548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:04:43.980144 26548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "6ea0ac3c05934f3ebc2022fecdeb7166"
format_stamp: "Formatted at 2026-08-11 09:04:37 on dist-test-slave-cbsf"
server_key: "0d19a9cb5838a7edd99037e136ac926c"
server_key_iv: "e7890f1fe88705621a479358b6e97e62"
server_key_version: "encryptionkey@0"
I20260811 09:04:43.981185 26548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:43.993923 26548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:43.997936 26548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:43.998168 26548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:44.018119 26548 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39877
I20260811 09:04:44.018143 26623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39877 every 8 connection(s)
I20260811 09:04:44.019701 26548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:04:44.024940 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26548
I20260811 09:04:44.026026 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:42785
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=44241
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:44.027818 26624 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:04:44.033179 26624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap starting.
I20260811 09:04:44.057336 26624 log.cc:824] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:44.075639 26624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:04:44.076344 26624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166: Bootstrap complete.
I20260811 09:04:44.086769 26624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:44.087769 26624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Initialized, Role: FOLLOWER
I20260811 09:04:44.088471 26624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:44.088761 26624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:04:44.088933 26624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:04:44.089143 26624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:04:44.093590 26624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:44.094168 26624 leader_election.cc:302] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ea0ac3c05934f3ebc2022fecdeb7166; no voters:
I20260811 09:04:44.094969 26624 leader_election.cc:288] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:04:44.095127 26629 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:04:44.096262 26629 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [term 3 LEADER]: Becoming Leader. State: Replica: 6ea0ac3c05934f3ebc2022fecdeb7166, State: Running, Role: LEADER
I20260811 09:04:44.096930 26629 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } }
I20260811 09:04:44.097472 26624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:04:44.099817 26630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:44.100346 26630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:44.102800 26631 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ea0ac3c05934f3ebc2022fecdeb7166. Latest consensus state: current_term: 3 leader_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ea0ac3c05934f3ebc2022fecdeb7166" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39877 } } }
I20260811 09:04:44.103169 26631 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ea0ac3c05934f3ebc2022fecdeb7166 [sys.catalog]: This master's current role is: LEADER
I20260811 09:04:44.105362 26635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:04:44.116842 26635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:04:44.119103 26635 catalog_manager.cc:1259] Loaded cluster ID: de5920351ae844bf824857b1f6d2bd0a
I20260811 09:04:44.119251 26635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 09:04:44.122589 26635 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f300a3759ad google::LogMessage::Fail() at ??:0
@ 0x7f300a37888a google::LogMessage::SendToLog() at ??:0
@ 0x7f300a3754de google::LogMessage::Flush() at ??:0
@ 0x7f300a376a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f301fda8512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f301fda5976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f301fe2d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f301fe2d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f300cfc1345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f300cfc491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f300cfc4799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f300cf9feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f30102796db start_thread at ??:0
@ 0x7f300920f61f clone at ??:0
W20260811 09:04:44.426047 26627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:44.426378 26627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:44.426558 26627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:44.434056 26627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:44.434203 26627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:44.434386 26627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:04:44.444446 26627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:42785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=44241
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:44.445812 26627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:44.447177 26627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:44.465961 26671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:44.466017 26674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:44.465921 26670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:44.472249 26627 server_base.cc:1034] running on GCE node
I20260811 09:04:44.473089 26627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:44.474058 26627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:44.475275 26627 hybrid_clock.cc:648] HybridClock initialized: now 1786439084475239 us; error 38 us; skew 500 ppm
I20260811 09:04:44.475628 26627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:44 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:44.478943 26627 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:04:44.480585 26627 webserver.cc:460] Webserver started at http://127.9.40.65:44241/ using document root <none> and password file <none>
I20260811 09:04:44.481360 26627 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:44.481496 26627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:44.486965 26627 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 09:04:44.490080 26681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:44.491459 26627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:04:44.491653 26627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "69f261710e3c408ca289d4f0fb994325"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "954a9a681d085d4e06848cfd3dbfc2ab"
server_key_iv: "b82d09d56363578b539c4a33f8a77f51"
server_key_version: "encryptionkey@0"
I20260811 09:04:44.492651 26627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:44.513091 26627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:44.517206 26627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:44.517496 26627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:44.518608 26627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:44.520475 26689 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 09:04:44.526299 26627 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 09:04:44.526414 26627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 09:04:44.526589 26627 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 09:04:44.529826 26627 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 09:04:44.529904 26627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.003s
I20260811 09:04:44.530308 26689 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Bootstrap starting.
I20260811 09:04:44.539263 26689 log.cc:824] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Log is configured to *not* fsync() on all Append() calls
I20260811 09:04:44.556605 26689 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:04:44.557370 26689 tablet_bootstrap.cc:492] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Bootstrap complete.
I20260811 09:04:44.557902 26689 ts_tablet_manager.cc:1397] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Time spent bootstrapping tablet: real 0.028s user 0.021s sys 0.005s
I20260811 09:04:44.568348 26689 raft_consensus.cc:348] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:44.570348 26689 raft_consensus.cc:729] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f261710e3c408ca289d4f0fb994325, State: Initialized, Role: FOLLOWER
I20260811 09:04:44.571602 26689 consensus_queue.cc:260] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:44.574067 26689 ts_tablet_manager.cc:1428] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260811 09:04:44.579672 26627 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42785
I20260811 09:04:44.579789 26813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42785 every 8 connection(s)
I20260811 09:04:44.581061 26627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:04:44.585858 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:44.583743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:53297)
0811 09:04:44.584647 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:44.584954 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":386,"threads_started":1}
W20260811 09:04:44.586568 26814 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39877 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:39877: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
I20260811 09:04:44.587414 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:44.587069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:43075)
0811 09:04:44.587207 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:44.587275 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:04:44.588203 26816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:44.587879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:47761)
0811 09:04:44.588011 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:44.588084 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:04:44.588457 26814 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:04:44.590339 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26627
I20260811 09:04:44.591050 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:45343
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=37593
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:04:44.940331 26818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:44.940620 26818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:44.940860 26818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:44.948273 26818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:44.948383 26818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:44.948601 26818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:04:44.958475 26818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:45343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=37593
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:44.959803 26818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:44.961274 26818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:44.974735 26823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:44.974756 26824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:44.974995 26826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:44.975639 26818 server_base.cc:1034] running on GCE node
I20260811 09:04:44.976253 26818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:44.977408 26818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:44.978637 26818 hybrid_clock.cc:648] HybridClock initialized: now 1786439084978593 us; error 40 us; skew 500 ppm
I20260811 09:04:44.978972 26818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:44 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:44.982371 26818 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:04:44.983857 26818 webserver.cc:460] Webserver started at http://127.9.40.66:37593/ using document root <none> and password file <none>
I20260811 09:04:44.984596 26818 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:44.984772 26818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:44.990702 26818 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 09:04:44.994853 26833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:44.997102 26818 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 09:04:44.997308 26818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "4cfc86a2dd7443de907d1762d474c539"
format_stamp: "Formatted at 2026-08-11 09:04:38 on dist-test-slave-cbsf"
server_key: "bef9dd80739e568e0dd7cb88b58b6c67"
server_key_iv: "fe09425ec738689def57180d9a17cd90"
server_key_version: "encryptionkey@0"
I20260811 09:04:44.998611 26818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:45.011044 26818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:45.015166 26818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:45.015479 26818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:45.016534 26818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:45.017976 26818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:45.018075 26818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:45.018231 26818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:45.018324 26818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:45.054057 26818 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45343
I20260811 09:04:45.054148 26961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45343 every 8 connection(s)
I20260811 09:04:45.055513 26818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:04:45.060550 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26818
I20260811 09:04:45.060593 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.058624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:44431)
0811 09:04:45.059362 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.059622 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":297,"threads_started":1}
W20260811 09:04:45.061245 26962 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39877 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:39877: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
I20260811 09:04:45.061374 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:38265
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:39877
--webserver_port=37053
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:04:45.062170 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.061857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:48005)
0811 09:04:45.062000 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.062066 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:04:45.063021 26964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.062682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:44819)
0811 09:04:45.062823 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.062907 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:04:45.063251 26962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:04:45.421739 26967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:04:45.422034 26967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:04:45.422224 26967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:04:45.429486 26967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:04:45.429600 26967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:04:45.429811 26967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:04:45.439587 26967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:38265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=37053
--tserver_master_addrs=127.9.40.126:39877
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:04:45.441011 26967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:04:45.442442 26967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:04:45.455782 26972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:45.455832 26973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:04:45.455888 26975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:04:45.457370 26967 server_base.cc:1034] running on GCE node
I20260811 09:04:45.458060 26967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:04:45.459000 26967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:04:45.460230 26967 hybrid_clock.cc:648] HybridClock initialized: now 1786439085460186 us; error 37 us; skew 500 ppm
I20260811 09:04:45.460628 26967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:04:45 dist-test-slave-cbsf krb5kdc[25330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439085, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:04:45.464298 26967 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:04:45.465984 26967 webserver.cc:460] Webserver started at http://127.9.40.67:37053/ using document root <none> and password file <none>
I20260811 09:04:45.466733 26967 fs_manager.cc:359] Metadata directory not provided
I20260811 09:04:45.466888 26967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:04:45.472468 26967 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:04:45.475263 26985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:45.476572 26967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:04:45.476824 26967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "43b77382b0fe4ac4833ca10aeac90985"
format_stamp: "Formatted at 2026-08-11 09:04:39 on dist-test-slave-cbsf"
server_key: "e12f86f8cdf293375ba33ffd358ad288"
server_key_iv: "0b289b12a5ba9fb6d4d11a088f83b848"
server_key_version: "encryptionkey@0"
I20260811 09:04:45.478072 26967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:04:45.497226 26967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:04:45.502105 26967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:04:45.502441 26967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:04:45.503628 26967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:04:45.505080 26967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:04:45.505215 26967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:45.505357 26967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:04:45.505452 26967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:04:45.541249 26967 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38265
I20260811 09:04:45.541356 27114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38265 every 8 connection(s)
I20260811 09:04:45.542850 26967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:04:45.544955 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 26967
I20260811 09:04:45.548491 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.546555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:38933)
0811 09:04:45.547363 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.547599 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":341,"threads_started":1}
W20260811 09:04:45.549194 27115 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:39877 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:39877: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
I20260811 09:04:45.550071 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.549726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:43057)
0811 09:04:45.549892 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.549959 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:04:45.550871 27119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.550553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:56099)
0811 09:04:45.550691 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.550768 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:04:45.551113 27115 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:04:45.589897 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.589092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:42595)
0811 09:04:45.589530 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.589666 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":200,"threads_started":1}
I20260811 09:04:45.798429 27121 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:45.798719 27121 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:45.800357 27121 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
Aug 11 09:04:45 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
Aug 11 09:04:45 dist-test-slave-cbsf krb5kdc[25330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439084, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:04:45.824002 27124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.801055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:37427 (local address 127.9.40.66:45343)
0811 09:04:45.802682 (+ 1627us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:45.802701 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:45.803603 (+ 902us) server_negotiation.cc:407] Connection header received
0811 09:04:45.805802 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:45.805811 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:45.806159 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:45.806478 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:45.808291 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.809545 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.811373 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.812006 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.812234 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:45.816798 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:45.816808 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:45.816823 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:45.816867 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:45.819971 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:45.820940 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:45.820956 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:45.820962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:45.821131 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:45.821592 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:45.821604 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:45.821607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:45.822140 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:45.822341 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:45.823173 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 09:04:45.823563 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1389,"thread_start_us":1205,"threads_started":1}
I20260811 09:04:45.824018 27122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.800681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45343 (local address 127.9.40.65:37427)
0811 09:04:45.803108 (+ 2427us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.803468 (+ 360us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:45.803615 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:45.807076 (+ 3461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:45.807086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:45.807613 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:45.808077 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:45.808098 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.809753 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.809756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:45.811222 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:45.811234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.812227 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.812234 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:45.812577 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:45.813429 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:45.813446 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:45.816439 (+ 2993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:45.820171 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:45.820180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:45.820196 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:45.820804 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:45.821262 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:45.821266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:45.821270 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:45.821467 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:45.822298 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:45.822306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:45.822644 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 09:04:45.822922 (+ 278us) client_negotiation.cc:240] Negotiation successful
0811 09:04:45.823508 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2298,"mutex_wait_us":195,"spinlock_wait_cycles":2944,"thread_start_us":2163,"threads_started":1}
I20260811 09:04:45.824011 27123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.801044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60595 (local address 127.9.40.67:38265)
0811 09:04:45.803111 (+ 2067us) server_negotiation.cc:206] Beginning negotiation
0811 09:04:45.803144 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 09:04:45.803196 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:04:45.805561 (+ 2365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:04:45.805576 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:04:45.806097 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:04:45.806478 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:04:45.808280 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.809603 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.811374 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.812016 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.812244 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:45.817022 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:04:45.817029 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:04:45.817040 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:04:45.817084 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:04:45.820044 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:45.820996 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:45.821008 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:45.821014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:45.821168 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:04:45.821635 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:04:45.821638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:04:45.821645 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:04:45.822140 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:04:45.822351 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 09:04:45.823170 (+ 819us) server_negotiation.cc:299] Negotiation successful
0811 09:04:45.823564 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1861,"thread_start_us":1654,"threads_started":1}
I20260811 09:04:45.824651 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:45.800627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:38265 (local address 127.9.40.65:60595)
0811 09:04:45.800962 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:45.801014 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:04:45.801744 (+ 730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:04:45.806618 (+ 4874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:04:45.806626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:04:45.807255 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:04:45.808078 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:45.808097 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.809783 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.809787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:45.811228 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:04:45.811240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:04:45.812283 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:04:45.812292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:04:45.812578 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:04:45.813382 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:04:45.813417 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:04:45.816762 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:04:45.820188 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:45.820196 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:45.820201 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:45.820844 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:45.821290 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:04:45.821296 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:04:45.821300 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:04:45.821489 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:04:45.822298 (+ 809us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:04:45.822306 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:04:45.822646 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 09:04:45.822922 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 09:04:45.823680 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":440,"spinlock_wait_cycles":52736}
W20260811 09:04:45.827596 26684 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
W20260811 09:04:45.827833 26686 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:45.828008 26684 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69f261710e3c408ca289d4f0fb994325; no voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539
I20260811 09:04:45.828409 27121 raft_consensus.cc:2726] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:04:46.065006 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:46.063891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:38591)
0811 09:04:46.064519 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:46.064672 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":339,"threads_started":1}
I20260811 09:04:46.552423 27126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:46.551669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:33319)
0811 09:04:46.552114 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:46.552223 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":227,"threads_started":1}
I20260811 09:04:46.591609 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:46.590794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:58381)
0811 09:04:46.591327 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:46.591430 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":220,"threads_started":1}
I20260811 09:04:47.066566 27129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:47.065889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:59597)
0811 09:04:47.066311 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:47.066392 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":223,"threads_started":1}
I20260811 09:04:47.554204 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:47.553454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:50303)
0811 09:04:47.553912 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:47.554006 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":237,"threads_started":1}
I20260811 09:04:47.593346 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:47.592565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:50839)
0811 09:04:47.593057 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:47.593157 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":260,"threads_started":1}
I20260811 09:04:48.060204 27132 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:48.060429 27132 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:48.061307 27132 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
W20260811 09:04:48.062307 26684 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
W20260811 09:04:48.062508 26686 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:48.062736 26686 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69f261710e3c408ca289d4f0fb994325; no voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539
I20260811 09:04:48.063084 27132 raft_consensus.cc:2726] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:04:48.068248 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:48.067473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:49383)
0811 09:04:48.067987 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:48.068077 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":287,"threads_started":1}
I20260811 09:04:48.556059 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:48.555216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:43929)
0811 09:04:48.555805 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:48.555901 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":362,"threads_started":1}
I20260811 09:04:48.595021 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:48.594267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:51449)
0811 09:04:48.594753 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:48.594848 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":253,"threads_started":1}
I20260811 09:04:49.069994 27136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:49.069204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:52899)
0811 09:04:49.069700 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:49.069789 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":240,"threads_started":1}
I20260811 09:04:49.557940 27137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:49.557105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:48897)
0811 09:04:49.557641 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:49.557737 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":250,"threads_started":1}
I20260811 09:04:49.596879 27138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:49.596010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:51157)
0811 09:04:49.596504 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:49.596600 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":226,"threads_started":1}
I20260811 09:04:50.071810 27139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:50.070982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:34437)
0811 09:04:50.071506 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:50.071619 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":330,"threads_started":1}
I20260811 09:04:50.559804 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:50.558917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:60579)
0811 09:04:50.559459 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:50.559584 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":291,"threads_started":1}
I20260811 09:04:50.598712 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:50.597919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:43981)
0811 09:04:50.598413 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:50.598501 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":276,"threads_started":1}
I20260811 09:04:51.073652 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:51.072682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:34431)
0811 09:04:51.073352 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:51.073436 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":286,"threads_started":1}
I20260811 09:04:51.313578 27143 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:51.313791 27143 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:51.314517 27143 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
W20260811 09:04:51.315618 26686 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
W20260811 09:04:51.315812 26684 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:51.315963 26684 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69f261710e3c408ca289d4f0fb994325; no voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539
I20260811 09:04:51.316220 27143 raft_consensus.cc:2726] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:04:51.561702 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:51.560880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:36701)
0811 09:04:51.561401 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:51.561487 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":285,"threads_started":1}
I20260811 09:04:51.600685 27145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:51.599833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:47739)
0811 09:04:51.600354 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:51.600473 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":270,"threads_started":1}
I20260811 09:04:52.075769 27146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:52.074611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:42379)
0811 09:04:52.075235 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:52.075499 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":277,"threads_started":1}
I20260811 09:04:52.563748 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:52.562798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:52377)
0811 09:04:52.563383 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:52.563517 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":263,"threads_started":1}
I20260811 09:04:52.602728 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:52.601774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:35399)
0811 09:04:52.602361 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:52.602484 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":322,"threads_started":1}
I20260811 09:04:53.077791 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:53.076871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:48073)
0811 09:04:53.077369 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:53.077488 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":241,"threads_started":1}
I20260811 09:04:53.565517 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:53.564789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:49651)
0811 09:04:53.565239 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:53.565319 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":237,"threads_started":1}
I20260811 09:04:53.604568 27152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:53.603703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:50683)
0811 09:04:53.604242 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:53.604362 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":301,"threads_started":1}
I20260811 09:04:54.079769 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:54.078851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:39713)
0811 09:04:54.079488 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:54.079587 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":270,"threads_started":1}
I20260811 09:04:54.567358 27154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:54.566554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:40405)
0811 09:04:54.567091 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:54.567187 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":260,"threads_started":1}
I20260811 09:04:54.606320 27155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:54.605597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:60063)
0811 09:04:54.606060 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:54.606143 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":248,"threads_started":1}
I20260811 09:04:54.808373 27156 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:54.808645 27156 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:54.809352 27156 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
W20260811 09:04:54.810418 26686 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
W20260811 09:04:54.810678 26684 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:54.810853 26684 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69f261710e3c408ca289d4f0fb994325; no voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539
I20260811 09:04:54.811127 27156 raft_consensus.cc:2726] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:04:55.082194 27157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:55.080976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:56037)
0811 09:04:55.081614 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:55.081711 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":368,"threads_started":1}
I20260811 09:04:55.569422 27158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:55.568519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:37095)
0811 09:04:55.569128 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:55.569231 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":357,"threads_started":1}
I20260811 09:04:55.608125 27159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:55.607317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:38041)
0811 09:04:55.607848 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:55.607936 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":301,"threads_started":1}
I20260811 09:04:56.084182 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:56.083291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:57883)
0811 09:04:56.083815 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:56.083962 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":272,"threads_started":1}
I20260811 09:04:56.571259 27162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:56.570481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:51525)
0811 09:04:56.570991 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:56.571086 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":232,"threads_started":1}
I20260811 09:04:56.610002 27163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:56.609226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:56373)
0811 09:04:56.609732 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:56.609822 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":281,"threads_started":1}
I20260811 09:04:57.086258 27164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:57.085311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:37171)
0811 09:04:57.085906 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:57.086033 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":261,"threads_started":1}
I20260811 09:04:57.464320 27166 raft_consensus.cc:482] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:04:57.464677 27166 raft_consensus.cc:504] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43b77382b0fe4ac4833ca10aeac90985" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 38265 } } peers { permanent_uuid: "4cfc86a2dd7443de907d1762d474c539" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45343 } } peers { permanent_uuid: "69f261710e3c408ca289d4f0fb994325" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42785 } }
I20260811 09:04:57.465708 27166 leader_election.cc:288] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265), 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343)
W20260811 09:04:57.466491 26686 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4cfc86a2dd7443de907d1762d474c539 (127.9.40.66:45343): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
W20260811 09:04:57.466660 26684 leader_election.cc:341] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 43b77382b0fe4ac4833ca10aeac90985 (127.9.40.67:38265): Not found: Tablet not found: 9bd60ea29e8f48e3bbebde82e6e556d4
I20260811 09:04:57.466749 26684 leader_election.cc:302] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 69f261710e3c408ca289d4f0fb994325; no voters: 43b77382b0fe4ac4833ca10aeac90985, 4cfc86a2dd7443de907d1762d474c539
I20260811 09:04:57.467006 27166 raft_consensus.cc:2726] T 9bd60ea29e8f48e3bbebde82e6e556d4 P 69f261710e3c408ca289d4f0fb994325 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:04:57.573163 27167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:57.572300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:41011)
0811 09:04:57.572884 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:57.572973 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":303,"threads_started":1}
I20260811 09:04:57.612043 27168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:57.611037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:60683)
0811 09:04:57.611590 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:57.611726 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":285,"threads_started":1}
I20260811 09:04:58.088399 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:58.087366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:39493)
0811 09:04:58.088106 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:58.088222 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":341,"threads_started":1}
I20260811 09:04:58.575253 27170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:58.574357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:60311)
0811 09:04:58.574925 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:58.575048 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":279,"threads_started":1}
I20260811 09:04:58.613821 27171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:58.613001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:37815)
0811 09:04:58.613495 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:58.613608 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":267,"threads_started":1}
I20260811 09:04:59.091049 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:59.089590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:50359)
0811 09:04:59.090648 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:59.090735 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":453,"threads_started":1}
I20260811 09:04:59.577319 27173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:59.576387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:56585)
0811 09:04:59.576982 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:59.577079 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":340,"threads_started":1}
I20260811 09:04:59.615923 27174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:04:59.615087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.65:55407)
0811 09:04:59.615646 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 09:04:59.615742 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":292,"threads_started":1}
I20260811 09:05:00.093111 27175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:00.092223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.66:49727)
0811 09:05:00.092832 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:00.092933 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
I20260811 09:05:00.549191 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26627
I20260811 09:05:00.567076 27177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:00.550672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40494 (local address 127.9.40.65:42785)
0811 09:05:00.551770 (+ 1098us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:00.551785 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:00.551826 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:05:00.551939 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:00.551953 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:00.552305 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:00.552491 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:00.554116 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.555148 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.556487 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.557283 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.557397 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:00.560297 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:00.560312 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:00.560330 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:00.560367 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:00.563880 (+ 3513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.564926 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.564933 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.564936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.565105 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.565510 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.565514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.565516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.566011 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:00.566328 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:00.566572 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 09:05:00.566786 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":695,"threads_started":1}
I20260811 09:05:00.568377 26809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:00.579584 27179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:00.578725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39877 (local address 127.9.40.67:43103)
0811 09:05:00.579287 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:00.579421 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:39877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":280,"threads_started":1}
I20260811 09:05:00.661937 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26627
I20260811 09:05:00.686244 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26818
I20260811 09:05:00.700902 27181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:00.687151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.9.40.66:45343)
0811 09:05:00.687698 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:00.687703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:00.687719 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:05:00.687777 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:00.687781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:00.687841 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:00.687964 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:00.689511 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.690296 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.691551 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.691993 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.692148 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:00.695123 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:00.695129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:00.695132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:00.695169 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:00.697966 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.698682 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.698689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.698693 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.698842 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.699269 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.699279 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.699281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.699566 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:00.699866 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:00.700190 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 09:05:00.700424 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":285,"threads_started":1}
I20260811 09:05:00.701690 26957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:00.789316 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26818
I20260811 09:05:00.814373 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 26967
I20260811 09:05:00.829605 27183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:00.815302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.9.40.67:38265)
0811 09:05:00.815913 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:00.815919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:00.815935 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:05:00.815993 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:00.815997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:00.816066 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:00.816197 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:00.817806 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.818634 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.819975 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:00.820438 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:00.820584 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:00.823493 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:00.823499 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:00.823502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:00.823547 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:00.826592 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.827353 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.827360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.827364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.827534 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:00.827938 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:00.827943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:00.827946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:00.828255 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:00.828596 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:00.829009 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 09:05:00.829269 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260811 09:05:00.830408 27109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:00.917397 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 26967
2026-08-11T09:05:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25324 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:05:01 dist-test-slave-cbsf krb5kdc[27188](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:05:01 dist-test-slave-cbsf krb5kdc[27188](info): set up 2 sockets
Aug 11 09:05:01 dist-test-slave-cbsf krb5kdc[27188](info): commencing operation
krb5kdc: starting...
W20260811 09:05:02.145247 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.006s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:05:02 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:05:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:05:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:05:02.462670 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:02.815770 27202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:02.816148 27202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:02.816329 27202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:02.823927 27202 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:05:02.824075 27202 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:05:02.824172 27202 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:05:02.824272 27202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:05:02.824385 27202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:02.824509 27202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:02.824609 27202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:05:02.824708 27202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:05:02.835364 27202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:02.836822 27202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:02.838234 27202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:02.850785 27207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:02.850818 27210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:02.850706 27208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:02.851750 27202 server_base.cc:1034] running on GCE node
I20260811 09:05:02.852653 27202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:02.854267 27202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:02.855543 27202 hybrid_clock.cc:648] HybridClock initialized: now 1786439102855473 us; error 36 us; skew 500 ppm
I20260811 09:05:02.855916 27202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:02 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:02.864485 27202 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:05:02.866170 27202 webserver.cc:460] Webserver started at http://127.9.40.126:46093/ using document root <none> and password file <none>
I20260811 09:05:02.866935 27202 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:02.867030 27202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:02.867304 27202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:05:02.870464 27202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "b939d131759c4a7ea7f4c48b3b6b0a35"
format_stamp: "Formatted at 2026-08-11 09:05:02 on dist-test-slave-cbsf"
server_key: "35507e6b11b08641b8d8d90218097e42"
server_key_iv: "7844af136608685098721ca7c9553d13"
server_key_version: "encryptionkey@0"
I20260811 09:05:02.871222 27202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "b939d131759c4a7ea7f4c48b3b6b0a35"
format_stamp: "Formatted at 2026-08-11 09:05:02 on dist-test-slave-cbsf"
server_key: "35507e6b11b08641b8d8d90218097e42"
server_key_iv: "7844af136608685098721ca7c9553d13"
server_key_version: "encryptionkey@0"
I20260811 09:05:02.876669 27202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:05:02.880645 27217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:02.882221 27202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:05:02.882396 27202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "b939d131759c4a7ea7f4c48b3b6b0a35"
format_stamp: "Formatted at 2026-08-11 09:05:02 on dist-test-slave-cbsf"
server_key: "35507e6b11b08641b8d8d90218097e42"
server_key_iv: "7844af136608685098721ca7c9553d13"
server_key_version: "encryptionkey@0"
I20260811 09:05:02.882623 27202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:02.918880 27202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:02.923682 27202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:02.923926 27202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:02.943835 27202 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38941
I20260811 09:05:02.943856 27276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38941 every 8 connection(s)
I20260811 09:05:02.945394 27202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:05:02.945881 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27202
I20260811 09:05:02.946173 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:05:02.947032 9377 external_mini_cluster.cc:1428] Setting key 1f7a54413b9aac6b92f2f32832235468
I20260811 09:05:02.950130 27277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:05:02 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:02.964427 27277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap starting.
I20260811 09:05:02.969094 27277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:02.970427 27277 log.cc:824] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:02.974243 27281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:02.949172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.9.40.126:38941)
0811 09:05:02.950017 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:02.950029 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:02.950071 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:05:02.951798 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:02.951822 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:02.952211 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:02.952563 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:02.954625 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:02.956097 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:02.958111 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:02.958958 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:02.959131 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:02.964544 (+ 5413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:02.964558 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:02.964580 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:02.964640 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:02.968873 (+ 4233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:02.969815 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:02.969824 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:02.969833 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:02.970080 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:02.970493 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:02.970498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:02.970501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:02.971433 (+ 932us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:02.971729 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:02.972347 (+ 618us) server_negotiation.cc:299] Negotiation successful
0811 09:05:02.972846 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":389,"threads_started":1}
I20260811 09:05:02.974367 27277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: No bootstrap required, opened a new log
I20260811 09:05:02.982481 27277 raft_consensus.cc:348] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:02.982887 27277 raft_consensus.cc:374] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:02.982990 27277 raft_consensus.cc:729] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Initialized, Role: FOLLOWER
I20260811 09:05:02.983647 27277 consensus_queue.cc:260] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:02.983891 27277 raft_consensus.cc:388] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:05:02.984030 27277 raft_consensus.cc:482] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:05:02.984210 27277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:02.986716 27277 raft_consensus.cc:504] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:02.987236 27277 leader_election.cc:302] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b939d131759c4a7ea7f4c48b3b6b0a35; no voters:
I20260811 09:05:02.987840 27277 leader_election.cc:288] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:05:02.987978 27283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:05:02.988682 27283 raft_consensus.cc:686] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 LEADER]: Becoming Leader. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Running, Role: LEADER
I20260811 09:05:02.989257 27283 consensus_queue.cc:237] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:02.989950 27277 sys_catalog.cc:564] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:05:02.992118 27284 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:02.992563 27284 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:02.993532 27285 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b939d131759c4a7ea7f4c48b3b6b0a35. Latest consensus state: current_term: 1 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:02.993875 27285 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:02.994118 27289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:05:03.003937 27289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:05:03.013749 27289 catalog_manager.cc:1347] Generated new cluster ID: 4c5dae3558364e2aa1353cfc958feebc
I20260811 09:05:03.013876 27289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:05:03.036616 27289 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:05:03.038121 27289 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:05:03.046020 27289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Generated new TSK 0
I20260811 09:05:03.046919 27289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:05:03.125388 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:03.523125 27309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:03.523438 27309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:03.523538 27309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:03.530838 27309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:03.530965 27309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:03.531081 27309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:05:03.541134 27309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:03.542538 27309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:03.543845 27309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:03.557682 27315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:03.557610 27318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:03.557658 27316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:03.558967 27309 server_base.cc:1034] running on GCE node
I20260811 09:05:03.559635 27309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:03.560629 27309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:03.561916 27309 hybrid_clock.cc:648] HybridClock initialized: now 1786439103561861 us; error 29 us; skew 500 ppm
I20260811 09:05:03.562295 27309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:03 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:03.565732 27309 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:05:03.567416 27309 webserver.cc:460] Webserver started at http://127.9.40.65:43121/ using document root <none> and password file <none>
I20260811 09:05:03.568137 27309 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:03.568280 27309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:03.568624 27309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:05:03.571709 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "4c7b9491adc64f22ba77984243fc4a56"
format_stamp: "Formatted at 2026-08-11 09:05:03 on dist-test-slave-cbsf"
server_key: "34d57006425d89f83b086cd0c4b3338c"
server_key_iv: "f0e00644d8e4d22f2d3ab4bab63068dd"
server_key_version: "encryptionkey@0"
I20260811 09:05:03.572526 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "4c7b9491adc64f22ba77984243fc4a56"
format_stamp: "Formatted at 2026-08-11 09:05:03 on dist-test-slave-cbsf"
server_key: "34d57006425d89f83b086cd0c4b3338c"
server_key_iv: "f0e00644d8e4d22f2d3ab4bab63068dd"
server_key_version: "encryptionkey@0"
I20260811 09:05:03.577976 27309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:05:03.581725 27326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:03.583130 27309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:05:03.583326 27309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "4c7b9491adc64f22ba77984243fc4a56"
format_stamp: "Formatted at 2026-08-11 09:05:03 on dist-test-slave-cbsf"
server_key: "34d57006425d89f83b086cd0c4b3338c"
server_key_iv: "f0e00644d8e4d22f2d3ab4bab63068dd"
server_key_version: "encryptionkey@0"
I20260811 09:05:03.583537 27309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:03.619618 27309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:03.624189 27309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:03.624583 27309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:03.625687 27309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:03.627111 27309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:03.627224 27309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:03.627389 27309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:03.627466 27309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:03.663606 27309 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37599
I20260811 09:05:03.663733 27450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37599 every 8 connection(s)
I20260811 09:05:03.665047 27309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:05:03.674391 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27309
I20260811 09:05:03.674649 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:05:03.675400 9377 external_mini_cluster.cc:1428] Setting key 1eff5a2c6877a3d2112246faee9919a6
Aug 11 09:05:03 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439103, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:03.687150 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:03.667798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:54999 (local address 127.9.40.126:38941)
0811 09:05:03.668370 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:03.668374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:03.669473 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 09:05:03.671373 (+ 1900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:03.671379 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:03.671486 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:03.671609 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:03.673286 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:03.674053 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:03.675850 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:03.676394 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:03.676535 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:03.680843 (+ 4308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:03.680848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:03.680852 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:03.680885 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:03.683570 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:03.684356 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:03.684365 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:03.684373 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:03.684501 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:03.685034 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:03.685039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:03.685041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:03.685328 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:03.685491 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:03.686354 (+ 863us) server_negotiation.cc:299] Negotiation successful
0811 09:05:03.686636 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":320,"threads_started":1}
I20260811 09:05:03.688194 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:03.668476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:54999)
0811 09:05:03.669267 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:03.669305 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:03.671098 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:03.671857 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:03.671866 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:03.672412 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:03.673107 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:03.673127 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:03.674231 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:03.674235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:03.675704 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:03.675717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:03.676716 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:03.676726 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:03.677023 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:03.678018 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:03.678047 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:03.680574 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:03.683725 (+ 3151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:03.683735 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:03.683754 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:03.684218 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:03.684621 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:03.684626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:03.684629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:03.684903 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:03.685488 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:03.685497 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:03.685876 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 09:05:03.686183 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 09:05:03.686710 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":349,"threads_started":1}
I20260811 09:05:03.689908 27452 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:03.690292 27452 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:03.691287 27452 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:03.693668 27237 ts_manager.cc:194] Registered new tserver with Master: 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599)
I20260811 09:05:03.695171 27237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:54999
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:05:03.744419 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:04.107474 27460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:04.107869 27460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:04.108052 27460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:04.115774 27460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:04.115947 27460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:04.116142 27460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:05:04.126787 27460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:04.128273 27460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:04.129721 27460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:04.144402 27466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:04.144390 27465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:04.144467 27469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:04.145460 27460 server_base.cc:1034] running on GCE node
I20260811 09:05:04.146227 27460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:04.147250 27460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:04.148458 27460 hybrid_clock.cc:648] HybridClock initialized: now 1786439104148385 us; error 66 us; skew 500 ppm
I20260811 09:05:04.148844 27460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:04 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439104, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:04.152408 27460 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:05:04.154065 27460 webserver.cc:460] Webserver started at http://127.9.40.66:38285/ using document root <none> and password file <none>
I20260811 09:05:04.154838 27460 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:04.154983 27460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:04.155344 27460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:05:04.158666 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "6463d91a2ba9b6fb83061b65087e3957"
server_key_iv: "fa10c033d24498ef07be0ab5ecc50950"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.159484 27460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "6463d91a2ba9b6fb83061b65087e3957"
server_key_iv: "fa10c033d24498ef07be0ab5ecc50950"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.165309 27460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:05:04.169487 27477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.171033 27460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:05:04.171330 27460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "6463d91a2ba9b6fb83061b65087e3957"
server_key_iv: "fa10c033d24498ef07be0ab5ecc50950"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.171598 27460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:04.190618 27460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:04.195254 27460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:04.195587 27460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:04.196677 27460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:04.198254 27460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:04.198333 27460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.198485 27460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:04.198554 27460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.233273 27460 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35247
I20260811 09:05:04.233367 27611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35247 every 8 connection(s)
I20260811 09:05:04.234588 27460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:05:04.238209 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27460
I20260811 09:05:04.238425 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:05:04.239086 9377 external_mini_cluster.cc:1428] Setting key 4e49f33001839cd1a92c314f2254137d
Aug 11 09:05:04 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439104, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:04.257644 27614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.237391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:53597 (local address 127.9.40.126:38941)
0811 09:05:04.237953 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.237961 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.239178 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 09:05:04.241299 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.241307 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.241394 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.241495 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:04.243855 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.244631 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.246515 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.246982 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.247136 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.251449 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:04.251454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:04.251458 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:04.251494 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:04.254298 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.255085 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.255093 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.255097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.255237 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.255696 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.255702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.255705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.256021 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:04.256184 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.256973 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.257245 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":242,"threads_started":1}
I20260811 09:05:04.258749 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.238041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:53597)
0811 09:05:04.238970 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:04.239029 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:04.241018 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:04.241823 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:04.241837 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:04.242713 (+ 876us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:04.243626 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.243656 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.244956 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.244961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.246349 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.246364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.247254 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.247263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.247613 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.248427 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:04.248451 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:04.251193 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:04.254462 (+ 3269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:04.254472 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:04.254491 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:04.254946 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:04.255357 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:04.255362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:04.255366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:04.255579 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:04.256157 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:04.256165 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:04.256538 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 09:05:04.256893 (+ 355us) client_negotiation.cc:240] Negotiation successful
0811 09:05:04.257499 (+ 606us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":374,"threads_started":1}
I20260811 09:05:04.260372 27612 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:04.260761 27612 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:04.261579 27612 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:04.263336 27237 ts_manager.cc:194] Registered new tserver with Master: 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247)
I20260811 09:05:04.264662 27237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:53597
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:05:04.313692 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:04.669234 27618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:04.669504 27618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:04.669603 27618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:04.677204 27618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:04.677331 27618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:04.677449 27618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:05:04.688091 27618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:04.689636 27618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:04.691126 27618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 09:05:04.698513 27452 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
W20260811 09:05:04.706187 27623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:04.706143 27624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:04.706233 27626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:04.710251 27618 server_base.cc:1034] running on GCE node
I20260811 09:05:04.710994 27618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:04.712101 27618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:04.713342 27618 hybrid_clock.cc:648] HybridClock initialized: now 1786439104713276 us; error 60 us; skew 500 ppm
I20260811 09:05:04.713699 27618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:04 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439104, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:04.717278 27618 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:05:04.719049 27618 webserver.cc:460] Webserver started at http://127.9.40.67:38655/ using document root <none> and password file <none>
I20260811 09:05:04.719882 27618 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:04.720046 27618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:04.720412 27618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:05:04.723757 27618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "29d78c049df4176dfcf8ffe9a0ac2319"
server_key_iv: "4355c35c3a9fdf07d88df066d7e3fb48"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.724645 27618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "29d78c049df4176dfcf8ffe9a0ac2319"
server_key_iv: "4355c35c3a9fdf07d88df066d7e3fb48"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.730528 27618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:05:04.734570 27633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.736123 27618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 09:05:04.736331 27618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "29d78c049df4176dfcf8ffe9a0ac2319"
server_key_iv: "4355c35c3a9fdf07d88df066d7e3fb48"
server_key_version: "encryptionkey@0"
I20260811 09:05:04.736585 27618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:04.759701 27618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:04.764539 27618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:04.764902 27618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:04.766003 27618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:04.767482 27618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:04.767596 27618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.767755 27618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:04.767831 27618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:04.803567 27618 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:34591
I20260811 09:05:04.803660 27760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:34591 every 8 connection(s)
I20260811 09:05:04.804968 27618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:05:04.807721 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27618
I20260811 09:05:04.807955 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:05:04.808539 9377 external_mini_cluster.cc:1428] Setting key 03fda62eb7de3d47d6d2d5c38a860933
Aug 11 09:05:04 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439104, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:04.827169 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.807884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53695 (local address 127.9.40.126:38941)
0811 09:05:04.809109 (+ 1225us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.809114 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.809957 (+ 843us) server_negotiation.cc:407] Connection header received
0811 09:05:04.812276 (+ 2319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.812280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.812353 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.812448 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:04.814086 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.814894 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.816462 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.816955 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.817069 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.820784 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:04.820789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:04.820793 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:04.820829 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:04.823877 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.824767 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.824772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.824776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.824958 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.825472 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.825476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.825479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.825795 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:04.825924 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.826676 (+ 752us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.826870 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1104,"thread_start_us":963,"threads_started":1}
I20260811 09:05:04.828207 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.808550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:53695)
0811 09:05:04.809782 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:04.809838 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:04.811991 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:04.812740 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:04.812755 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:04.813308 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:04.813897 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.813920 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.815035 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.815039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.816320 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.816333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.817191 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.817196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.817401 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.818142 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:04.818166 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:04.820513 (+ 2347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:04.824039 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:04.824052 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:04.824086 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:04.824575 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:04.825099 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:04.825102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:04.825105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:04.825315 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:04.825931 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:04.825938 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:04.826310 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 09:05:04.826562 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 09:05:04.826985 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":930,"thread_start_us":670,"threads_started":1}
I20260811 09:05:04.829689 27761 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:04.830036 27761 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:04.830791 27761 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:04.832257 27237 ts_manager.cc:194] Registered new tserver with Master: dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
I20260811 09:05:04.833176 27237 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:53695
I20260811 09:05:04.837244 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:05:04.855773 27764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.842138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.9.40.126:38941)
0811 09:05:04.842316 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.842320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.842868 (+ 548us) server_negotiation.cc:407] Connection header received
0811 09:05:04.843025 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.843028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.843098 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.843180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:04.844783 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.845530 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.846863 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.847253 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.847345 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.849937 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:04.849942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:04.849946 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:04.849979 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:04.853309 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.853978 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.853982 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.853985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.854107 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:04.854415 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:04.854418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:04.854420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:04.854749 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:04.854944 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.855334 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.855510 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:05:04.859980 27237 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:05:04.867178 27237 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:05:04.892630 27777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.880194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41192)
0811 09:05:04.881896 (+ 1702us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:04.881957 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:04.882132 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:04.884502 (+ 2370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:04.884512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:04.884574 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:04.885207 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.885221 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.887386 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.887391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.889623 (+ 2232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.889636 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.891529 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.891537 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.891729 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.891784 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 09:05:04.891895 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:05:04.892096 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"mutex_wait_us":53,"thread_start_us":2234,"threads_started":1}
I20260811 09:05:04.893292 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.880200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34591 (local address 127.0.0.1:37444)
0811 09:05:04.882750 (+ 2550us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:04.883615 (+ 865us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:04.883752 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:04.884810 (+ 1058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:04.884815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:04.884836 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:04.885375 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.885389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.887383 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.887388 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.890254 (+ 2866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.890269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.892248 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.892254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.892434 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.892475 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 09:05:04.892586 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:05:04.892840 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2432,"mutex_wait_us":219,"spinlock_wait_cycles":19328,"thread_start_us":2272,"threads_started":1}
I20260811 09:05:04.893473 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.881574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35247 (local address 127.0.0.1:56444)
0811 09:05:04.884873 (+ 3299us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:04.884891 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:04.885080 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:04.885693 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:04.885697 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:04.885725 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:04.886154 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.886168 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.888324 (+ 2156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.888330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.890609 (+ 2279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:04.890623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.892599 (+ 1976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.892605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:04.892803 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.892835 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:05:04.892915 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:05:04.893066 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3192,"mutex_wait_us":3,"thread_start_us":2895,"threads_started":1}
I20260811 09:05:04.894068 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.882095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56444 (local address 127.9.40.66:35247)
0811 09:05:04.884276 (+ 2181us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.884287 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.885042 (+ 755us) server_negotiation.cc:407] Connection header received
0811 09:05:04.885272 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.885280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.885488 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.885697 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:04.886323 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.888167 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.890806 (+ 2639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.892094 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.892236 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.892426 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.893694 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.893823 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2017,"thread_start_us":1916,"threads_started":1}
I20260811 09:05:04.894088 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.880871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41192 (local address 127.9.40.65:37599)
0811 09:05:04.883893 (+ 3022us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.883901 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.883925 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:05:04.884006 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.884012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.884259 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.885287 (+ 1028us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:04.885554 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.887239 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.889828 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.891306 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.893212 (+ 1906us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.893444 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.893535 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.893729 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2842,"thread_start_us":2668,"threads_started":1}
I20260811 09:05:04.895440 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:04.880306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.9.40.67:34591)
0811 09:05:04.883528 (+ 3222us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:04.883539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:04.883889 (+ 350us) server_negotiation.cc:407] Connection header received
0811 09:05:04.883996 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:04.884004 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:04.884260 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:04.884453 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:04.885579 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.887186 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.890418 (+ 3232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:04.892017 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:04.892144 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:04.892392 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:04.892677 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 09:05:04.894025 (+ 1348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3037,"thread_start_us":2877,"threads_started":1}
I20260811 09:05:04.898329 27528 tablet_service.cc:1467] Processing CreateTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 (DEFAULT_TABLE table=test-table [id=1c4376cb6ce44bd99a08e1a4f1b06e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:04.898327 27379 tablet_service.cc:1467] Processing CreateTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 (DEFAULT_TABLE table=test-table [id=1c4376cb6ce44bd99a08e1a4f1b06e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:04.898521 27690 tablet_service.cc:1467] Processing CreateTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 (DEFAULT_TABLE table=test-table [id=1c4376cb6ce44bd99a08e1a4f1b06e19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:04.899919 27690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf802b4f4a304e1fa7d3c74a76e946e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:04.899921 27379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf802b4f4a304e1fa7d3c74a76e946e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:04.899915 27528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf802b4f4a304e1fa7d3c74a76e946e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:04.912303 27784 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Bootstrap starting.
I20260811 09:05:04.914747 27786 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Bootstrap starting.
I20260811 09:05:04.916095 27785 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Bootstrap starting.
I20260811 09:05:04.916507 27784 tablet_bootstrap.cc:654] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:04.918291 27784 log.cc:824] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:04.919862 27786 tablet_bootstrap.cc:654] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:04.920258 27785 tablet_bootstrap.cc:654] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:04.921016 27786 log.cc:824] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:04.921777 27785 log.cc:824] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:04.924407 27786 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: No bootstrap required, opened a new log
I20260811 09:05:04.924777 27785 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: No bootstrap required, opened a new log
I20260811 09:05:04.924787 27786 ts_tablet_manager.cc:1397] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 09:05:04.925051 27785 ts_tablet_manager.cc:1397] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:05:04.925503 27784 tablet_bootstrap.cc:492] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: No bootstrap required, opened a new log
I20260811 09:05:04.925788 27784 ts_tablet_manager.cc:1397] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 09:05:04.932934 27785 raft_consensus.cc:348] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.933322 27785 raft_consensus.cc:374] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:04.933429 27785 raft_consensus.cc:729] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7b9491adc64f22ba77984243fc4a56, State: Initialized, Role: FOLLOWER
I20260811 09:05:04.934056 27785 consensus_queue.cc:260] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.935885 27785 ts_tablet_manager.cc:1428] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 09:05:04.935742 27786 raft_consensus.cc:348] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.936158 27786 raft_consensus.cc:374] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:04.936288 27786 raft_consensus.cc:729] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3e3c70f6ef42798e54495927f3d6ea, State: Initialized, Role: FOLLOWER
I20260811 09:05:04.936542 27784 raft_consensus.cc:348] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.936996 27784 raft_consensus.cc:374] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:04.937114 27784 raft_consensus.cc:729] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacf6dfe62e94e4cbfad31e775a84dff, State: Initialized, Role: FOLLOWER
I20260811 09:05:04.937000 27786 consensus_queue.cc:260] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.937742 27784 consensus_queue.cc:260] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:04.938284 27612 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
I20260811 09:05:04.939043 27786 ts_tablet_manager.cc:1428] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 09:05:04.940441 27761 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
I20260811 09:05:04.941121 27784 ts_tablet_manager.cc:1428] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
W20260811 09:05:04.986850 27613 tablet.cc:2369] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:05:05.044389 27794 raft_consensus.cc:482] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:05:05.044777 27794 raft_consensus.cc:504] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:05.046646 27794 leader_election.cc:288] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599), 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247)
I20260811 09:05:05.053447 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.047470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:51541 (local address 127.9.40.66:35247)
0811 09:05:05.047662 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.047666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.047682 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:05:05.047745 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.047748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.047806 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.047914 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:05.048374 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.049511 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.051382 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.052491 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.052602 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.052675 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.053115 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.053217 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:05:05.053522 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.047032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35247 (local address 127.9.40.67:51541)
0811 09:05:05.047234 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.047260 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:05.047400 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:05.047954 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:05.047958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:05.047993 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:05.048209 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:05.048223 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.049681 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.049687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:05.051206 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:05.051218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.052674 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.052682 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:05.052916 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.052956 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:05:05.053081 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 09:05:05.053222 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 09:05:05.054458 27797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.047049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.9.40.67:48521)
0811 09:05:05.048004 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.048019 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:05.048138 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:05.048505 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:05.048512 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:05.048539 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:05.048971 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:05.048984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.050270 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.050274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:05.051926 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:05.051941 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.053552 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.053561 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:05.053898 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.053934 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:05:05.054047 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 09:05:05.054185 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":40,"thread_start_us":540,"threads_started":1}
I20260811 09:05:05.054504 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.047080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48521 (local address 127.9.40.65:37599)
0811 09:05:05.047348 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.047357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.048127 (+ 770us) server_negotiation.cc:407] Connection header received
0811 09:05:05.048266 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.048274 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.048371 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.048447 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:05.049135 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.050092 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.052147 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.053317 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.053454 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.053577 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.054088 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.054238 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:05:05.054795 27555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf802b4f4a304e1fa7d3c74a76e946e9" candidate_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" is_pre_election: true
I20260811 09:05:05.055439 27555 raft_consensus.cc:2454] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dacf6dfe62e94e4cbfad31e775a84dff in term 0.
I20260811 09:05:05.055413 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf802b4f4a304e1fa7d3c74a76e946e9" candidate_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7b9491adc64f22ba77984243fc4a56" is_pre_election: true
I20260811 09:05:05.055982 27402 raft_consensus.cc:2454] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dacf6dfe62e94e4cbfad31e775a84dff in term 0.
I20260811 09:05:05.056275 27638 leader_election.cc:302] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b3e3c70f6ef42798e54495927f3d6ea, dacf6dfe62e94e4cbfad31e775a84dff; no voters:
I20260811 09:05:05.056800 27794 raft_consensus.cc:2781] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260811 09:05:05.056806 27762 tablet.cc:2369] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:05:05.056977 27794 raft_consensus.cc:482] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:05:05.057117 27794 raft_consensus.cc:3037] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:05.059419 27794 raft_consensus.cc:504] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:05.060137 27794 leader_election.cc:288] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [CANDIDATE]: Term 1 election: Requested vote from peers 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599), 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247)
I20260811 09:05:05.060612 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf802b4f4a304e1fa7d3c74a76e946e9" candidate_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7b9491adc64f22ba77984243fc4a56"
I20260811 09:05:05.060686 27555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf802b4f4a304e1fa7d3c74a76e946e9" candidate_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
I20260811 09:05:05.060868 27402 raft_consensus.cc:3037] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:05.060954 27555 raft_consensus.cc:3037] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:05.063181 27402 raft_consensus.cc:2454] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dacf6dfe62e94e4cbfad31e775a84dff in term 1.
I20260811 09:05:05.063359 27555 raft_consensus.cc:2454] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dacf6dfe62e94e4cbfad31e775a84dff in term 1.
I20260811 09:05:05.063627 27634 leader_election.cc:302] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56, dacf6dfe62e94e4cbfad31e775a84dff; no voters:
I20260811 09:05:05.063930 27794 raft_consensus.cc:2781] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:05:05.064378 27794 raft_consensus.cc:686] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 1 LEADER]: Becoming Leader. State: Replica: dacf6dfe62e94e4cbfad31e775a84dff, State: Running, Role: LEADER
I20260811 09:05:05.064944 27794 consensus_queue.cc:237] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } }
I20260811 09:05:05.070073 27236 catalog_manager.cc:5495] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff reported cstate change: term changed from 0 to 1, leader changed from <none> to dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67). New cstate: current_term: 1 leader_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } health_report { overall_health: UNKNOWN } } }
I20260811 09:05:05.134756 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.126396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37456 (local address 127.9.40.67:34591)
0811 09:05:05.126648 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.126660 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.126678 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:05:05.126776 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.126779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.126871 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.127054 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:05:05.127517 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.128318 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.132349 (+ 4031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.132857 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.133016 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.133587 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:05:05.134148 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:05:05.134320 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.134379 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.134472 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:05:05.143658 27555 raft_consensus.cc:1261] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Refusing update from remote peer dacf6dfe62e94e4cbfad31e775a84dff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:05:05.144681 27798 consensus_queue.cc:1035] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:05:05.145067 27402 raft_consensus.cc:1261] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Refusing update from remote peer dacf6dfe62e94e4cbfad31e775a84dff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:05:05.145879 27798 consensus_queue.cc:1035] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:05:05.157761 27801 mvcc.cc:205] Tried to move back new op lower bound from 7317254574657794048 to 7317254574350077952. Current Snapshot: MvccSnapshot[applied={T|T < 7317254574657794048}]
I20260811 09:05:05.173908 27237 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:05:05.174235 27237 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:05:05.178452 27237 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Sending DeleteTablet for 3 replicas of tablet cf802b4f4a304e1fa7d3c74a76e946e9
I20260811 09:05:05.179769 27690 tablet_service.cc:1514] Processing DeleteTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37444
I20260811 09:05:05.179827 27528 tablet_service.cc:1514] Processing DeleteTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:56444
I20260811 09:05:05.180343 27379 tablet_service.cc:1514] Processing DeleteTablet for tablet cf802b4f4a304e1fa7d3c74a76e946e9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:41192
I20260811 09:05:05.185416 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27309
I20260811 09:05:05.196866 27813 tablet_replica.cc:330] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: stopping tablet replica
I20260811 09:05:05.197436 27813 raft_consensus.cc:2229] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:05:05.198055 27813 raft_consensus.cc:2258] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:05:05.199018 27812 tablet_replica.cc:330] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: stopping tablet replica
I20260811 09:05:05.199510 27812 raft_consensus.cc:2229] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:05:05.200263 27812 raft_consensus.cc:2258] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:05:05.202857 27812 ts_tablet_manager.cc:1905] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:05:05.205375 27814 tablet_replica.cc:330] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: stopping tablet replica
I20260811 09:05:05.205889 27814 raft_consensus.cc:2229] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:05:05.206580 27814 raft_consensus.cc:2258] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:05:05.206892 27813 ts_tablet_manager.cc:1905] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:05:05 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:05:05.211045 27814 ts_tablet_manager.cc:1905] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:05:05.211110 27812 ts_tablet_manager.cc:1918] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:05.211314 27812 log.cc:1196] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/cf802b4f4a304e1fa7d3c74a76e946e9
I20260811 09:05:05.211947 27812 ts_tablet_manager.cc:1939] T cf802b4f4a304e1fa7d3c74a76e946e9 P dacf6dfe62e94e4cbfad31e775a84dff: Deleting consensus metadata
I20260811 09:05:05.214082 27220 catalog_manager.cc:4841] TS dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): tablet cf802b4f4a304e1fa7d3c74a76e946e9 (table test-table [id=1c4376cb6ce44bd99a08e1a4f1b06e19]) successfully deleted
I20260811 09:05:05.214403 27813 ts_tablet_manager.cc:1918] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:05.214589 27813 log.cc:1196] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/cf802b4f4a304e1fa7d3c74a76e946e9
I20260811 09:05:05.215162 27813 ts_tablet_manager.cc:1939] T cf802b4f4a304e1fa7d3c74a76e946e9 P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting consensus metadata
I20260811 09:05:05.217178 27222 catalog_manager.cc:4841] TS 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): tablet cf802b4f4a304e1fa7d3c74a76e946e9 (table test-table [id=1c4376cb6ce44bd99a08e1a4f1b06e19]) successfully deleted
I20260811 09:05:05.221529 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.187098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.9.40.65:37599)
0811 09:05:05.187340 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.187347 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.188976 (+ 1629us) server_negotiation.cc:407] Connection header received
0811 09:05:05.190138 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.190147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.190260 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.190368 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:05.196044 (+ 5676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.197393 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.200904 (+ 3511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.201643 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.201767 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.214504 (+ 12737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:05.214521 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:05.214543 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:05.214606 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:05.217966 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.218967 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.218975 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.218978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.219139 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.219593 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.219598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.219600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.220013 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:05.220387 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.220790 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.221059 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:05:05.222321 27446 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:05.331389 27814 ts_tablet_manager.cc:1918] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:05.331924 27814 log.cc:1196] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/cf802b4f4a304e1fa7d3c74a76e946e9
I20260811 09:05:05.332134 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27309
W20260811 09:05:05.357429 27219 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37599 (error 108)
I20260811 09:05:05.357614 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27460
I20260811 09:05:05.359426 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.358904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41210)
0811 09:05:05.359108 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.359238 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 09:05:05.359730 27219 catalog_manager.cc:4584] TS 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf802b4f4a304e1fa7d3c74a76e946e9: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Aug 11 09:05:05 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:05:05.375352 27783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.358988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56448 (local address 127.9.40.66:35247)
0811 09:05:05.359170 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.359175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.359191 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:05:05.359248 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.359252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.359313 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.359453 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:05.361047 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.361805 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.363287 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.363700 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.363851 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.368641 (+ 4790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:05.368660 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:05.368678 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:05.368775 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:05.372016 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.372960 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.372969 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.372971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.373118 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.373551 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.373555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.373557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.374003 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:05.374346 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.374703 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.374946 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:05:05.376073 27606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:05.413816 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.413252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41212)
0811 09:05:05.413506 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.413632 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 09:05:05.415124 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.414658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41222)
0811 09:05:05.414840 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.414955 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:05:05.471324 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27460
I20260811 09:05:05.476013 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.475309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41228)
0811 09:05:05.475641 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.475771 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 09:05:05.477341 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.476929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41236)
0811 09:05:05.477118 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.477207 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:05:05.506071 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27618
Aug 11 09:05:05 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:05:05.525943 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.506980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37464 (local address 127.9.40.67:34591)
0811 09:05:05.507204 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:05.507210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:05.507237 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:05:05.507412 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:05.507426 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:05.507518 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:05.507634 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:05.509286 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.510529 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.511963 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:05.512717 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:05.512873 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:05.517900 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:05.517922 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:05.517946 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:05.517999 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:05.522370 (+ 4371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.523279 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.523291 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.523295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.523550 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:05.523946 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:05.523952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:05.523956 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:05.524530 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:05.525038 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:05.525216 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:05:05.525504 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:05:05.526870 27756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:05.558205 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.557622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41242)
0811 09:05:05.557877 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.557988 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 09:05:05.559785 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.559094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41258)
0811 09:05:05.559329 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.559435 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 09:05:05.659241 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27618
I20260811 09:05:05.685468 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27202
I20260811 09:05:05.686925 27271 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:05.804188 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27202
I20260811 09:05:05.804256 27782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:05.803568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:41264)
0811 09:05:05.803833 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:05.803965 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:05:25.834883 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--webserver_interface=127.9.40.126
--webserver_port=46093
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:26.208854 27823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:26.209240 27823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:26.209416 27823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:26.218712 27823 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:05:26.218848 27823 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:05:26.218917 27823 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:05:26.218997 27823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:05:26.219084 27823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:26.219190 27823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:26.219282 27823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:05:26.219376 27823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:05:26.235127 27823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=46093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:26.237183 27823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:26.239040 27823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:26.271797 27835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:26.271903 27831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:26.271870 27832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:26.274360 27823 server_base.cc:1034] running on GCE node
I20260811 09:05:26.275347 27823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:26.277176 27823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:26.278402 27823 hybrid_clock.cc:648] HybridClock initialized: now 1786439126278336 us; error 30 us; skew 500 ppm
I20260811 09:05:26.278880 27823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:26 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:26.282912 27823 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:05:26.284914 27823 webserver.cc:460] Webserver started at http://127.9.40.126:46093/ using document root <none> and password file <none>
I20260811 09:05:26.285778 27823 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:26.285948 27823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:26.292124 27823 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 09:05:26.295301 27842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:26.296675 27823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:05:26.296945 27823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "b939d131759c4a7ea7f4c48b3b6b0a35"
format_stamp: "Formatted at 2026-08-11 09:05:02 on dist-test-slave-cbsf"
server_key: "35507e6b11b08641b8d8d90218097e42"
server_key_iv: "7844af136608685098721ca7c9553d13"
server_key_version: "encryptionkey@0"
I20260811 09:05:26.298081 27823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:26.314057 27823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:26.319041 27823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:26.319383 27823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:26.340934 27823 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38941
I20260811 09:05:26.340955 27899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38941 every 8 connection(s)
I20260811 09:05:26.342636 27823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:05:26.345105 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27823
I20260811 09:05:26.346196 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:37599
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=43121
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:05:26.353101 27900 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:05:26.358770 27900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap starting.
I20260811 09:05:26.380491 27900 log.cc:824] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:26.393493 27900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:05:26.394116 27900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap complete.
I20260811 09:05:26.405519 27900 raft_consensus.cc:348] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:26.406525 27900 raft_consensus.cc:729] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Initialized, Role: FOLLOWER
I20260811 09:05:26.407194 27900 consensus_queue.cc:260] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:26.407456 27900 raft_consensus.cc:388] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:05:26.407588 27900 raft_consensus.cc:482] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:05:26.407780 27900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:05:26.411962 27900 raft_consensus.cc:504] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:26.412575 27900 leader_election.cc:302] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b939d131759c4a7ea7f4c48b3b6b0a35; no voters:
I20260811 09:05:26.413305 27900 leader_election.cc:288] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:05:26.413460 27905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:05:26.414745 27905 raft_consensus.cc:686] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 LEADER]: Becoming Leader. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Running, Role: LEADER
I20260811 09:05:26.415366 27905 consensus_queue.cc:237] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:26.415771 27900 sys_catalog.cc:564] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:05:26.418375 27907 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b939d131759c4a7ea7f4c48b3b6b0a35. Latest consensus state: current_term: 2 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:26.418797 27907 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:26.418445 27906 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:26.418928 27906 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:26.421350 27912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:05:26.436849 27912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:05:26.438512 27912 catalog_manager.cc:1259] Loaded cluster ID: 4c5dae3558364e2aa1353cfc958feebc
I20260811 09:05:26.438750 27912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:05:26.443176 27912 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:05:26.444932 27912 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Loaded TSK: 0
I20260811 09:05:26.491111 27912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Generated new TSK 1
I20260811 09:05:26.491582 27912 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:05:26.756871 27902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:26.757272 27902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:26.757444 27902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:26.765378 27902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:26.765550 27902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:26.765751 27902 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:05:26.777040 27902 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:37599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=43121
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:26.778646 27902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:26.780153 27902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:26.796380 27936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:26.796257 27933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:26.796589 27934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:26.797190 27902 server_base.cc:1034] running on GCE node
I20260811 09:05:26.797961 27902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:26.799044 27902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:26.800262 27902 hybrid_clock.cc:648] HybridClock initialized: now 1786439126800200 us; error 55 us; skew 500 ppm
I20260811 09:05:26.800624 27902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:26 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:26.804276 27902 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:05:26.805974 27902 webserver.cc:460] Webserver started at http://127.9.40.65:43121/ using document root <none> and password file <none>
I20260811 09:05:26.806712 27902 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:26.806864 27902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:26.812620 27902 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:05:26.815721 27943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:26.817466 27902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:05:26.817669 27902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "4c7b9491adc64f22ba77984243fc4a56"
format_stamp: "Formatted at 2026-08-11 09:05:03 on dist-test-slave-cbsf"
server_key: "34d57006425d89f83b086cd0c4b3338c"
server_key_iv: "f0e00644d8e4d22f2d3ab4bab63068dd"
server_key_version: "encryptionkey@0"
I20260811 09:05:26.818729 27902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:26.844820 27902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:26.849298 27902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:26.849625 27902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:26.850744 27902 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:26.852849 27952 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 09:05:26.858610 27952 ts_tablet_manager.cc:1852] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 09:05:26.859746 27952 ts_tablet_manager.cc:1905] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:05:26.869473 27952 ts_tablet_manager.cc:1918] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:26.869911 27952 ts_tablet_manager.cc:1939] T cf802b4f4a304e1fa7d3c74a76e946e9 P 4c7b9491adc64f22ba77984243fc4a56: Deleting consensus metadata
I20260811 09:05:26.870447 27902 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 09:05:26.870668 27902 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260811 09:05:26.870836 27902 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:26.870935 27902 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:26.907902 27902 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37599
I20260811 09:05:26.907951 28079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37599 every 8 connection(s)
I20260811 09:05:26.909294 27902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:05:26.910277 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 27902
I20260811 09:05:26.911091 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:35247
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=38285
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:05:26 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439126, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:26.937978 28085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:26.912825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:40601)
0811 09:05:26.913822 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:26.913879 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:26.915771 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:26.917623 (+ 1852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:26.917631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:26.918176 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:26.918774 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:26.918793 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:26.921369 (+ 2576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:26.921373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:26.922828 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:26.922844 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:26.924460 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:26.924471 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:26.924862 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:26.925794 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:26.925853 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:26.928850 (+ 2997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:26.933013 (+ 4163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:26.933024 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:26.933043 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:26.933579 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:26.934110 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:26.934115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:26.934118 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:26.934322 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:26.935186 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:26.935195 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:26.935560 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 09:05:26.935870 (+ 310us) client_negotiation.cc:240] Negotiation successful
0811 09:05:26.936510 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":455,"threads_started":1}
I20260811 09:05:26.937966 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:26.912615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40601 (local address 127.9.40.126:38941)
0811 09:05:26.913319 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:26.913337 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:26.914048 (+ 711us) server_negotiation.cc:407] Connection header received
0811 09:05:26.916474 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:26.916495 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:26.916966 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:26.917438 (+ 472us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:26.919002 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:26.921131 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:26.923024 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:26.924188 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:26.924336 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:26.929228 (+ 4892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:26.929238 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:26.929255 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:26.929301 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:26.932811 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:26.933760 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:26.933769 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:26.933775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:26.933963 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:26.934449 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:26.934455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:26.934460 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:26.935011 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:26.935235 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:26.936106 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 09:05:26.936509 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":263,"threads_started":1}
I20260811 09:05:26.941744 28081 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:26.942155 28081 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:26.943230 28081 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:26.945847 27860 ts_manager.cc:194] Registered new tserver with Master: 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599)
I20260811 09:05:26.947355 27860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:40601
W20260811 09:05:27.283473 28086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:27.283767 28086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:27.283928 28086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:27.291563 28086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:27.291705 28086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:27.291968 28086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:05:27.303656 28086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:35247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=38285
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:27.305166 28086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:27.306666 28086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:27.321573 28092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:27.321609 28091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:27.321722 28094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:27.322469 28086 server_base.cc:1034] running on GCE node
I20260811 09:05:27.323185 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:27.324383 28086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:27.325649 28086 hybrid_clock.cc:648] HybridClock initialized: now 1786439127325586 us; error 49 us; skew 500 ppm
I20260811 09:05:27.325996 28086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:27 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:27.329496 28086 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:05:27.331120 28086 webserver.cc:460] Webserver started at http://127.9.40.66:38285/ using document root <none> and password file <none>
I20260811 09:05:27.331866 28086 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:27.332041 28086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:27.337939 28086 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:05:27.341233 28103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.342731 28086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:05:27.342932 28086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "6463d91a2ba9b6fb83061b65087e3957"
server_key_iv: "fa10c033d24498ef07be0ab5ecc50950"
server_key_version: "encryptionkey@0"
I20260811 09:05:27.343986 28086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:27.369694 28086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:27.374369 28086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:27.374774 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:27.375984 28086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:27.377765 28086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:27.377919 28086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.378068 28086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:27.378181 28086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.416189 28086 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35247
I20260811 09:05:27.416363 28244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35247 every 8 connection(s)
I20260811 09:05:27.417840 28086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:05:27.427923 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28086
I20260811 09:05:27.428669 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:34591
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=38655
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:05:27 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:27.440804 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.420890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:37449 (local address 127.9.40.126:38941)
0811 09:05:27.421106 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.421113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.422414 (+ 1301us) server_negotiation.cc:407] Connection header received
0811 09:05:27.424612 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.424620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.424775 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.424898 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:27.426521 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.427265 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.429088 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.429568 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.429711 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.433991 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:27.433997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:27.434001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:27.434037 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:27.436922 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.437805 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.437816 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.437819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.437950 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.438665 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.438671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.438674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.439064 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:27.439224 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:27.440144 (+ 920us) server_negotiation.cc:299] Negotiation successful
0811 09:05:27.440413 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:05:27.442127 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.421365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:37449)
0811 09:05:27.422215 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:27.422257 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:27.424308 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:27.425164 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:27.425172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:27.425732 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:27.426339 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:27.426357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.427439 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.427444 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:27.428930 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:27.428949 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.429829 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.429837 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:27.430081 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.430841 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:27.430864 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:27.433692 (+ 2828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:27.437101 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:27.437111 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:27.437131 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:27.437651 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:27.438076 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:27.438083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:27.438086 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:27.438332 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:27.439332 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:27.439339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:27.439724 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 09:05:27.440031 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 09:05:27.440643 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":362,"threads_started":1}
I20260811 09:05:27.443859 28245 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:27.444217 28245 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:27.445058 28245 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:27.446646 27860 ts_manager.cc:194] Registered new tserver with Master: 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247)
I20260811 09:05:27.447494 27860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:37449
W20260811 09:05:27.790869 28249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:27.791203 28249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:27.791396 28249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:27.798816 28249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:27.798983 28249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:27.799182 28249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:05:27.809631 28249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:34591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=38655
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:27.811048 28249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:27.812546 28249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:27.826903 28257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:27.826853 28260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:27.826872 28258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:27.827805 28249 server_base.cc:1034] running on GCE node
I20260811 09:05:27.828459 28249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:27.829573 28249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:27.830837 28249 hybrid_clock.cc:648] HybridClock initialized: now 1786439127830780 us; error 40 us; skew 500 ppm
I20260811 09:05:27.831221 28249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:27 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:27.834667 28249 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:05:27.836298 28249 webserver.cc:460] Webserver started at http://127.9.40.67:38655/ using document root <none> and password file <none>
I20260811 09:05:27.837237 28249 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:27.837409 28249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:27.843202 28249 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:05:27.846060 28269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.847612 28249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:05:27.847790 28249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "29d78c049df4176dfcf8ffe9a0ac2319"
server_key_iv: "4355c35c3a9fdf07d88df066d7e3fb48"
server_key_version: "encryptionkey@0"
I20260811 09:05:27.848886 28249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:27.866650 28249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:27.871101 28249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:27.871330 28249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:27.872313 28249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:27.873831 28249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:27.873925 28249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.874002 28249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:27.874070 28249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:27.909780 28249 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:34591
I20260811 09:05:27.909806 28394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:34591 every 8 connection(s)
I20260811 09:05:27.911242 28249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:05:27.914423 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28249
Aug 11 09:05:27 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439127, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:05:27.933581 28401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.915354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50266 (local address 127.9.40.126:38941)
0811 09:05:27.916217 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.916222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.917369 (+ 1147us) server_negotiation.cc:407] Connection header received
0811 09:05:27.917580 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.917584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.917649 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.917739 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:27.919962 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.920867 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.922738 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.923164 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.923298 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.927222 (+ 3924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:27.927226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:27.927230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:27.927260 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:27.930310 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.931356 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.931360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.931362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.931485 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.931937 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.931940 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.931942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.932243 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:27.932534 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:27.932925 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 09:05:27.933165 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":586,"threads_started":1}
I20260811 09:05:27.935781 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.914145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57099 (local address 127.9.40.126:38941)
0811 09:05:27.914389 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.914396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.915631 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 09:05:27.918618 (+ 2987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.918621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.918672 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.918747 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:27.920768 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.921670 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.923931 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.924603 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.924760 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.928927 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:27.928933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:27.928936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:27.928980 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:27.932168 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.932984 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.932990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.932994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.933130 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.933681 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.933686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.933690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.934021 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:27.934204 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:27.935189 (+ 985us) server_negotiation.cc:299] Negotiation successful
0811 09:05:27.935407 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:05:27.936983 28399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.914584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:57099)
0811 09:05:27.915436 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:27.915490 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:27.918314 (+ 2824us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:27.919064 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:27.919078 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:27.919845 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:27.920576 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:27.920596 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.921839 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.921845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:27.923771 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:27.923789 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.924881 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.924890 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:27.925269 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.925986 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:27.926022 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:27.928651 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:27.932310 (+ 3659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:27.932327 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:27.932342 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:27.932866 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:27.933246 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:27.933252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:27.933255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:27.933550 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:27.934203 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:27.934212 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:27.934730 (+ 518us) client_negotiation.cc:769] Sending connection context
0811 09:05:27.935106 (+ 376us) client_negotiation.cc:240] Negotiation successful
0811 09:05:27.935677 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":364,"threads_started":1}
I20260811 09:05:27.938350 28396 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38941
I20260811 09:05:27.938766 28396 heartbeater.cc:464] Registering TS with master...
I20260811 09:05:27.939553 28396 heartbeater.cc:511] Master 127.9.40.126:38941 requested a full tablet report, sending...
I20260811 09:05:27.941184 27860 ts_manager.cc:194] Registered new tserver with Master: dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
I20260811 09:05:27.941978 27860 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:57099
I20260811 09:05:27.949010 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:05:27.951407 28081 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
I20260811 09:05:27.956305 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.949923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50276 (local address 127.9.40.126:38941)
0811 09:05:27.950145 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.950154 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.950680 (+ 526us) server_negotiation.cc:407] Connection header received
0811 09:05:27.950860 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.950863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.951003 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.951204 (+ 201us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:05:27.951649 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.952564 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.954228 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.954664 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.954804 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.955325 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:05:27.955623 (+ 298us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 09:05:27.956015 (+ 392us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50276: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:05:27.956432 28083 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50276: token expired
W20260811 09:05:27.957053 27769 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.40.126:38941)
I20260811 09:05:27.970543 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.957637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50280 (local address 127.9.40.126:38941)
0811 09:05:27.957799 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.957805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.957842 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:05:27.958023 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.958050 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.958121 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.958223 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:27.959586 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.960304 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.961837 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:27.962251 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:27.962389 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:27.965227 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:27.965232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:27.965235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:27.965267 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:27.967690 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.968388 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.968394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.968398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.968529 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:27.968981 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:27.968990 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:27.968992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:27.969327 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:27.969602 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:27.969953 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 09:05:27.970209 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:05:27.974177 27860 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:05:27.981640 27860 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:05:28.012056 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.994421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.9.40.67:34591)
0811 09:05:27.998081 (+ 3660us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:27.998094 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:27.998140 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:05:27.998260 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:27.998271 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:27.998595 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:27.999763 (+ 1168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:28.000114 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.003777 (+ 3663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.009131 (+ 5354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.010733 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.010870 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.011124 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.011457 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.011651 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2176,"thread_start_us":2025,"threads_started":1}
I20260811 09:05:28.012071 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.994223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34591 (local address 127.0.0.1:39038)
0811 09:05:27.996077 (+ 1854us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:27.996094 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:27.996197 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:27.998865 (+ 2668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:27.998876 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:27.998916 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:27.999484 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:27.999501 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.003996 (+ 4495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.004004 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.007442 (+ 3438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.007470 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.010969 (+ 3499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.010976 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.011197 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.011247 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 09:05:28.011382 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 09:05:28.011585 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1739,"thread_start_us":683,"threads_started":1}
I20260811 09:05:28.016508 28410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.993814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35247 (local address 127.0.0.1:56638)
0811 09:05:27.995265 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:27.995304 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:27.995480 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:28.009347 (+ 13867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:28.009352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:28.009379 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:28.009781 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.009794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.011630 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.011636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.013945 (+ 2309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.013960 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.015649 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.015656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.015826 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.015860 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:05:28.015962 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:05:28.016103 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1210,"mutex_wait_us":29,"thread_start_us":2012,"threads_started":1}
I20260811 09:05:28.016572 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.994027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.9.40.66:35247)
0811 09:05:28.008580 (+ 14553us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.008589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.008622 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:05:28.008755 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.008766 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.009086 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.009367 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:28.009990 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.011467 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.014155 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.015397 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.015533 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.015739 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.016038 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.016223 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14361,"thread_start_us":14229,"threads_started":1}
I20260811 09:05:28.016848 28324 tablet_service.cc:1467] Processing CreateTablet for tablet 95795d17289147be851138c41328409f (DEFAULT_TABLE table=test-table [id=c17d8998c778421c8feb0b89ba86ff15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:28.017944 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.997304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48594 (local address 127.9.40.65:37599)
0811 09:05:28.009092 (+ 11788us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.009101 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.009126 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:05:28.009223 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.009229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.009436 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.009635 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:28.010278 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.011626 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.014782 (+ 3156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.016377 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.016518 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.017460 (+ 942us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.017535 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.017693 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11601,"thread_start_us":11409,"threads_started":1}
I20260811 09:05:28.018494 28324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95795d17289147be851138c41328409f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:28.018643 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:27.996754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48594)
0811 09:05:28.008179 (+ 11425us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.008206 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:28.008369 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:28.009678 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:28.009684 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:28.009718 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:28.010104 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.010112 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.011806 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.011812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.014601 (+ 2789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.014617 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.016823 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.016830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.017027 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.017061 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:05:28.017155 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:05:28.018236 (+ 1081us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11202,"mutex_wait_us":1,"thread_start_us":10996,"threads_started":1}
I20260811 09:05:28.020931 28166 tablet_service.cc:1467] Processing CreateTablet for tablet 95795d17289147be851138c41328409f (DEFAULT_TABLE table=test-table [id=c17d8998c778421c8feb0b89ba86ff15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:28.021087 28004 tablet_service.cc:1467] Processing CreateTablet for tablet 95795d17289147be851138c41328409f (DEFAULT_TABLE table=test-table [id=c17d8998c778421c8feb0b89ba86ff15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:05:28.022320 28166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95795d17289147be851138c41328409f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:28.022313 28004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95795d17289147be851138c41328409f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:05:28.032307 28417 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Bootstrap starting.
I20260811 09:05:28.034461 28418 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Bootstrap starting.
I20260811 09:05:28.037284 28417 tablet_bootstrap.cc:654] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:28.038244 28418 tablet_bootstrap.cc:654] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:28.038484 28419 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Bootstrap starting.
I20260811 09:05:28.038681 28417 log.cc:824] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:28.039841 28418 log.cc:824] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:28.042181 28419 tablet_bootstrap.cc:654] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Neither blocks nor log segments found. Creating new log.
I20260811 09:05:28.043099 28418 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: No bootstrap required, opened a new log
I20260811 09:05:28.043375 28418 ts_tablet_manager.cc:1397] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:05:28.043440 28417 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: No bootstrap required, opened a new log
I20260811 09:05:28.043468 28419 log.cc:824] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:28.043766 28417 ts_tablet_manager.cc:1397] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260811 09:05:28.046772 28419 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: No bootstrap required, opened a new log
I20260811 09:05:28.047070 28419 ts_tablet_manager.cc:1397] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260811 09:05:28.051961 28418 raft_consensus.cc:348] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.052389 28418 raft_consensus.cc:374] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:28.052539 28418 raft_consensus.cc:729] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7b9491adc64f22ba77984243fc4a56, State: Initialized, Role: FOLLOWER
I20260811 09:05:28.053216 28418 consensus_queue.cc:260] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.055264 28418 ts_tablet_manager.cc:1428] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 09:05:28.055099 28417 raft_consensus.cc:348] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.055619 28417 raft_consensus.cc:374] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:28.055752 28417 raft_consensus.cc:729] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dacf6dfe62e94e4cbfad31e775a84dff, State: Initialized, Role: FOLLOWER
I20260811 09:05:28.056504 28417 consensus_queue.cc:260] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.058346 28396 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
I20260811 09:05:28.058576 28417 ts_tablet_manager.cc:1428] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 09:05:28.060225 28419 raft_consensus.cc:348] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.060796 28419 raft_consensus.cc:374] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:05:28.060900 28419 raft_consensus.cc:729] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3e3c70f6ef42798e54495927f3d6ea, State: Initialized, Role: FOLLOWER
I20260811 09:05:28.061655 28419 consensus_queue.cc:260] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.062893 28245 heartbeater.cc:503] Master 127.9.40.126:38941 was elected leader, sending a full tablet report...
I20260811 09:05:28.063544 28419 ts_tablet_manager.cc:1428] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Time spent starting tablet: real 0.016s user 0.011s sys 0.002s
I20260811 09:05:28.131686 28425 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:05:28.132006 28425 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.133766 28425 leader_election.cc:288] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591), 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599)
I20260811 09:05:28.141263 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.134272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:47221 (local address 127.9.40.65:37599)
0811 09:05:28.134507 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.134514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.135506 (+ 992us) server_negotiation.cc:407] Connection header received
0811 09:05:28.135750 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.135759 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.135822 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.135923 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:28.136478 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.137308 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.139353 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.140230 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.140386 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.140508 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.140844 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.140985 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:05:28.141381 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.134162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:48759 (local address 127.9.40.67:34591)
0811 09:05:28.134442 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.134449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.135616 (+ 1167us) server_negotiation.cc:407] Connection header received
0811 09:05:28.135840 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.135845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.135909 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.136015 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:05:28.136717 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.137457 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.139400 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.140300 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.140458 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.140574 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.140916 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.141051 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 09:05:28.141518 28427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.134412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.9.40.66:47221)
0811 09:05:28.135365 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.135381 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:28.135595 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:28.135939 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:28.135944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:28.135967 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:28.136323 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.136333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.137451 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.137457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.139154 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.139191 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.140472 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.140478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.140640 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.140665 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:05:28.140814 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 09:05:28.140967 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"mutex_wait_us":1,"thread_start_us":557,"threads_started":1}
I20260811 09:05:28.141574 28428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.134498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34591 (local address 127.9.40.66:48759)
0811 09:05:28.135481 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.135497 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:28.135680 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:28.136038 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:28.136042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:28.136085 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:05:28.136524 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.136533 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.137619 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.137631 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.139253 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:28.139266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.140533 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.140539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:28.140754 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.140787 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:05:28.140888 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:05:28.141019 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":824,"mutex_wait_us":19,"thread_start_us":548,"threads_started":1}
I20260811 09:05:28.142673 28350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95795d17289147be851138c41328409f" candidate_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" is_pre_election: true
I20260811 09:05:28.142674 28029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95795d17289147be851138c41328409f" candidate_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7b9491adc64f22ba77984243fc4a56" is_pre_election: true
I20260811 09:05:28.143301 28029 raft_consensus.cc:2454] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3e3c70f6ef42798e54495927f3d6ea in term 0.
I20260811 09:05:28.143299 28350 raft_consensus.cc:2454] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3e3c70f6ef42798e54495927f3d6ea in term 0.
I20260811 09:05:28.144081 28104 leader_election.cc:302] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56, 8b3e3c70f6ef42798e54495927f3d6ea; no voters:
I20260811 09:05:28.144599 28425 raft_consensus.cc:2781] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:05:28.144804 28425 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:05:28.144923 28425 raft_consensus.cc:3037] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:28.147298 28425 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.147907 28425 leader_election.cc:288] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [CANDIDATE]: Term 1 election: Requested vote from peers dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591), 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599)
I20260811 09:05:28.148483 28029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95795d17289147be851138c41328409f" candidate_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c7b9491adc64f22ba77984243fc4a56"
I20260811 09:05:28.148524 28350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95795d17289147be851138c41328409f" candidate_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
I20260811 09:05:28.148746 28029 raft_consensus.cc:3037] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:28.148746 28350 raft_consensus.cc:3037] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:05:28.152140 28350 raft_consensus.cc:2454] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3e3c70f6ef42798e54495927f3d6ea in term 1.
I20260811 09:05:28.152140 28029 raft_consensus.cc:2454] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3e3c70f6ef42798e54495927f3d6ea in term 1.
I20260811 09:05:28.152568 28104 leader_election.cc:302] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56, 8b3e3c70f6ef42798e54495927f3d6ea; no voters:
I20260811 09:05:28.152913 28425 raft_consensus.cc:2781] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:05:28.153458 28425 raft_consensus.cc:686] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 LEADER]: Becoming Leader. State: Replica: 8b3e3c70f6ef42798e54495927f3d6ea, State: Running, Role: LEADER
I20260811 09:05:28.154002 28425 consensus_queue.cc:237] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:28.159256 27858 catalog_manager.cc:5495] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66). New cstate: current_term: 1 leader_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } health_report { overall_health: UNKNOWN } } }
W20260811 09:05:28.161989 28082 tablet.cc:2369] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:05:28.163103 28398 tablet.cc:2369] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:05:28.170250 28246 tablet.cc:2369] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:05:28.201977 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.196256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.9.40.66:35247)
0811 09:05:28.196523 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.196538 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.196559 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:05:28.196808 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.196812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.196912 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.197063 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:05:28.197411 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.198205 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.199631 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.200073 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.200178 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.200681 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:05:28.201300 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:05:28.201444 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.201580 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.201651 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 09:05:28.210716 28029 raft_consensus.cc:1261] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Refusing update from remote peer 8b3e3c70f6ef42798e54495927f3d6ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:05:28.210717 28350 raft_consensus.cc:1261] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Refusing update from remote peer 8b3e3c70f6ef42798e54495927f3d6ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:05:28.211764 28429 consensus_queue.cc:1035] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:05:28.212144 28425 consensus_queue.cc:1035] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:05:28.224591 28432 mvcc.cc:205] Tried to move back new op lower bound from 7317254669139931136 to 7317254668923383808. Current Snapshot: MvccSnapshot[applied={T|T < 7317254669139931136}]
I20260811 09:05:28.226140 28433 mvcc.cc:205] Tried to move back new op lower bound from 7317254669139931136 to 7317254668923383808. Current Snapshot: MvccSnapshot[applied={T|T < 7317254669139931136}]
I20260811 09:05:28.238596 27859 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:05:28.238909 27859 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:05:28.242974 27859 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Sending DeleteTablet for 3 replicas of tablet 95795d17289147be851138c41328409f
I20260811 09:05:28.244402 28324 tablet_service.cc:1514] Processing DeleteTablet for tablet 95795d17289147be851138c41328409f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:28 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:39038
I20260811 09:05:28.244402 28166 tablet_service.cc:1514] Processing DeleteTablet for tablet 95795d17289147be851138c41328409f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:28 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:56638
I20260811 09:05:28.244630 28004 tablet_service.cc:1514] Processing DeleteTablet for tablet 95795d17289147be851138c41328409f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:05:28 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48594
I20260811 09:05:28.245541 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27902
I20260811 09:05:28.277860 28415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.246969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.9.40.65:37599)
0811 09:05:28.247244 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.247254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.247279 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:05:28.247375 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.247383 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.247519 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.247656 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:28.250505 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.252063 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.255019 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.255990 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.256155 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.261831 (+ 5676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:28.261860 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:28.261892 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:28.261959 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:28.267541 (+ 5582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.270207 (+ 2666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.270221 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.270227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.270476 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.271642 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.271650 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.271655 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.272323 (+ 668us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:28.275265 (+ 2942us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.276038 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.276374 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 09:05:28.278807 28074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:28.286172 28445 tablet_replica.cc:330] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: stopping tablet replica
I20260811 09:05:28.286545 28444 tablet_replica.cc:330] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: stopping tablet replica
I20260811 09:05:28.287469 28445 raft_consensus.cc:2229] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:05:28.288225 28445 raft_consensus.cc:2258] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:05:28.288715 28444 raft_consensus.cc:2229] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:05:28.289258 28444 raft_consensus.cc:2258] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:05:28.291455 28444 ts_tablet_manager.cc:1905] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:05:28.294119 28445 ts_tablet_manager.cc:1905] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:05:28.299381 28444 ts_tablet_manager.cc:1918] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:28.299582 28444 log.cc:1196] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/95795d17289147be851138c41328409f
I20260811 09:05:28.300222 28444 ts_tablet_manager.cc:1939] T 95795d17289147be851138c41328409f P dacf6dfe62e94e4cbfad31e775a84dff: Deleting consensus metadata
I20260811 09:05:28.302430 28445 ts_tablet_manager.cc:1918] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:05:28.302450 27844 catalog_manager.cc:4841] TS dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): tablet 95795d17289147be851138c41328409f (table test-table [id=c17d8998c778421c8feb0b89ba86ff15]) successfully deleted
I20260811 09:05:28.302742 28445 log.cc:1196] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/95795d17289147be851138c41328409f
I20260811 09:05:28.303416 28445 ts_tablet_manager.cc:1939] T 95795d17289147be851138c41328409f P 8b3e3c70f6ef42798e54495927f3d6ea: Deleting consensus metadata
I20260811 09:05:28.305341 27845 catalog_manager.cc:4841] TS 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): tablet 95795d17289147be851138c41328409f (table test-table [id=c17d8998c778421c8feb0b89ba86ff15]) successfully deleted
I20260811 09:05:28.383292 28446 tablet_replica.cc:330] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: stopping tablet replica
I20260811 09:05:28.383625 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27902
I20260811 09:05:28.409893 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 28086
W20260811 09:05:28.409947 27843 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37599 (error 108)
I20260811 09:05:28.412026 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.411513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48618)
0811 09:05:28.411717 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.411844 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 09:05:28.412403 27843 catalog_manager.cc:4584] TS 4c7b9491adc64f22ba77984243fc4a56 (127.9.40.65:37599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95795d17289147be851138c41328409f: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
I20260811 09:05:28.426999 28409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.411600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.9.40.66:35247)
0811 09:05:28.411786 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.411792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.411811 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:05:28.411884 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.411889 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.411988 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.412112 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:28.413865 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.414736 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.416195 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.416982 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.417119 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.419950 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:28.419963 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:28.419977 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:28.420026 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:28.423615 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.424441 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.424451 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.424454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.424617 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.425184 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.425189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.425192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.425617 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:28.425959 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.426312 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.426559 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:05:28.427798 28240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:28.466332 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.465748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48622)
0811 09:05:28.466072 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.466174 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 09:05:28.467523 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.467146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48630)
0811 09:05:28.467305 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.467404 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:05:28.528196 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.527709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48636)
0811 09:05:28.527941 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.528035 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 09:05:28.529372 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.529048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48640)
0811 09:05:28.529186 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.529258 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:05:28.547963 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 28086
I20260811 09:05:28.572381 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 28249
I20260811 09:05:28.586282 28413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.573236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39052 (local address 127.9.40.67:34591)
0811 09:05:28.573412 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:28.573417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:28.573434 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:05:28.573582 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:28.573587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:28.573656 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:28.573747 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:28.575237 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.576016 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.577514 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:28.577920 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:28.578027 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:28.580767 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:28.580785 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:28.580804 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:28.580856 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:28.583550 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.584262 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.584270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.584272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.584418 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:28.584927 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:28.584930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:28.584932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:28.585307 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:28.585529 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:28.585822 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:05:28.585990 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:05:28.586953 28390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:28.610544 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.609860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48642)
0811 09:05:28.610153 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.610276 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 09:05:28.611765 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.611290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48648)
0811 09:05:28.611514 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.611624 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 09:05:28.681151 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 28249
I20260811 09:05:28.703701 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 27823
I20260811 09:05:28.704972 27894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:05:28.823622 28414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:28.822880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37599 (local address 127.0.0.1:48662)
0811 09:05:28.823062 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:28.823245 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:05:28.823904 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 27823
I20260811 09:05:48.850903 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--webserver_interface=127.9.40.126
--webserver_port=46093
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:49.228148 28453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:49.228544 28453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:49.228745 28453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:49.236430 28453 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:05:49.236588 28453 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:05:49.236663 28453 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:05:49.236788 28453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:05:49.236881 28453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:49.237008 28453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:49.237114 28453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:05:49.237246 28453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:05:49.248044 28453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=46093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:49.249648 28453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:49.251173 28453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:49.265390 28458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:49.265349 28459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:49.265442 28461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:49.266438 28453 server_base.cc:1034] running on GCE node
I20260811 09:05:49.267340 28453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:49.268929 28453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:49.270190 28453 hybrid_clock.cc:648] HybridClock initialized: now 1786439149270125 us; error 42 us; skew 500 ppm
I20260811 09:05:49.270550 28453 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:49 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439149, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:49.273950 28453 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:05:49.275521 28453 webserver.cc:460] Webserver started at http://127.9.40.126:46093/ using document root <none> and password file <none>
I20260811 09:05:49.276321 28453 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:49.276472 28453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:49.282353 28453 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 09:05:49.285413 28469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:49.286798 28453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:05:49.287009 28453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "b939d131759c4a7ea7f4c48b3b6b0a35"
format_stamp: "Formatted at 2026-08-11 09:05:02 on dist-test-slave-cbsf"
server_key: "35507e6b11b08641b8d8d90218097e42"
server_key_iv: "7844af136608685098721ca7c9553d13"
server_key_version: "encryptionkey@0"
I20260811 09:05:49.288022 28453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:49.313004 28453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:49.317873 28453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:49.318174 28453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:49.338619 28453 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38941
I20260811 09:05:49.338752 28526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38941 every 8 connection(s)
I20260811 09:05:49.340394 28453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:05:49.347736 28527 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:05:49.348351 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28453
I20260811 09:05:49.349470 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:37599
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=43121
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:05:49.353070 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap starting.
I20260811 09:05:49.377883 28527 log.cc:824] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:49.396625 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:05:49.397480 28527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Bootstrap complete.
I20260811 09:05:49.407272 28527 raft_consensus.cc:348] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:49.408214 28527 raft_consensus.cc:729] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Initialized, Role: FOLLOWER
I20260811 09:05:49.408993 28527 consensus_queue.cc:260] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:49.409240 28527 raft_consensus.cc:388] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:05:49.409359 28527 raft_consensus.cc:482] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:05:49.409554 28527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:05:49.413589 28527 raft_consensus.cc:504] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:49.414147 28527 leader_election.cc:302] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b939d131759c4a7ea7f4c48b3b6b0a35; no voters:
I20260811 09:05:49.414901 28527 leader_election.cc:288] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:05:49.415046 28531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:05:49.416471 28531 raft_consensus.cc:686] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [term 3 LEADER]: Becoming Leader. State: Replica: b939d131759c4a7ea7f4c48b3b6b0a35, State: Running, Role: LEADER
I20260811 09:05:49.417099 28531 consensus_queue.cc:237] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } }
I20260811 09:05:49.417498 28527 sys_catalog.cc:564] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:05:49.420266 28533 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:49.420238 28534 sys_catalog.cc:455] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b939d131759c4a7ea7f4c48b3b6b0a35. Latest consensus state: current_term: 3 leader_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b939d131759c4a7ea7f4c48b3b6b0a35" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38941 } } }
I20260811 09:05:49.420737 28534 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:49.420738 28533 sys_catalog.cc:458] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35 [sys.catalog]: This master's current role is: LEADER
I20260811 09:05:49.434408 28539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:05:49.443547 28539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:05:49.445180 28539 catalog_manager.cc:1259] Loaded cluster ID: 4c5dae3558364e2aa1353cfc958feebc
I20260811 09:05:49.445317 28539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:05:49.449731 28539 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:05:49.451457 28539 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Loaded TSK: 0
I20260811 09:05:49.451618 28539 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b939d131759c4a7ea7f4c48b3b6b0a35: Loaded TSK: 1
F20260811 09:05:49.452800 28539 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f79b8ff29ad google::LogMessage::Fail() at ??:0
@ 0x7f79b8ff588a google::LogMessage::SendToLog() at ??:0
@ 0x7f79b8ff24de google::LogMessage::Flush() at ??:0
@ 0x7f79b8ff3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f79bcd567d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f79bcd4573e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f79cea211c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f79cea20783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f79ceaac2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f79ceaac0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f79ceaf65e9 std::function<>::operator()() at ??:0
@ 0x7f79cea24cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f79cea22c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f79ceaaa610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f79ceaaa439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f79bbc3e345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f79bbc4191c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f79bbc41799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f79bbc1ceaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f79beef66db start_thread at ??:0
@ 0x7f79b7e8c61f clone at ??:0
W20260811 09:05:49.777817 28529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:49.778184 28529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:49.778365 28529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:49.786233 28529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:49.786419 28529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:49.786621 28529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:05:49.797166 28529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:37599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=43121
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:49.798705 28529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:49.800158 28529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:49.814369 28587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:49.814304 28589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:49.814379 28591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:49.815294 28529 server_base.cc:1034] running on GCE node
I20260811 09:05:49.816032 28529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:49.817183 28529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:49.818423 28529 hybrid_clock.cc:648] HybridClock initialized: now 1786439149818379 us; error 53 us; skew 500 ppm
I20260811 09:05:49.818773 28529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:49 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439149, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:49.822165 28529 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:05:49.823748 28529 webserver.cc:460] Webserver started at http://127.9.40.65:43121/ using document root <none> and password file <none>
I20260811 09:05:49.824539 28529 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:49.824654 28529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:49.830222 28529 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:05:49.833261 28598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:49.834677 28529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:05:49.834823 28529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "4c7b9491adc64f22ba77984243fc4a56"
format_stamp: "Formatted at 2026-08-11 09:05:03 on dist-test-slave-cbsf"
server_key: "34d57006425d89f83b086cd0c4b3338c"
server_key_iv: "f0e00644d8e4d22f2d3ab4bab63068dd"
server_key_version: "encryptionkey@0"
I20260811 09:05:49.835813 28529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:49.855636 28529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:49.859992 28529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:49.860242 28529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:49.861322 28529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:49.863318 28606 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 09:05:49.869895 28529 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 09:05:49.870127 28529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260811 09:05:49.870329 28529 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 09:05:49.873984 28529 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 09:05:49.874094 28529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20260811 09:05:49.874504 28606 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Bootstrap starting.
I20260811 09:05:49.904681 28606 log.cc:824] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Log is configured to *not* fsync() on all Append() calls
I20260811 09:05:49.916986 28529 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37599
I20260811 09:05:49.916991 28728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37599 every 8 connection(s)
I20260811 09:05:49.918443 28529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:05:49.919502 28606 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:05:49.920380 28606 tablet_bootstrap.cc:492] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Bootstrap complete.
I20260811 09:05:49.922017 28606 ts_tablet_manager.cc:1397] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Time spent bootstrapping tablet: real 0.048s user 0.030s sys 0.015s
I20260811 09:05:49.924609 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:49.921839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:57893)
0811 09:05:49.923094 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:49.923412 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":676,"threads_started":1}
W20260811 09:05:49.925618 28729 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:38941 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:38941: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
I20260811 09:05:49.926577 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28529
I20260811 09:05:49.927150 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:49.926506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:57765)
0811 09:05:49.926879 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:49.926990 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235}
I20260811 09:05:49.927342 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:35247
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=38285
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:05:49.929296 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:49.927798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:54089)
0811 09:05:49.928671 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:49.928946 (+ 275us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":367,"threads_started":1}
W20260811 09:05:49.929628 28729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:05:49.931392 28606 raft_consensus.cc:348] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:49.932459 28606 raft_consensus.cc:729] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c7b9491adc64f22ba77984243fc4a56, State: Initialized, Role: FOLLOWER
I20260811 09:05:49.933192 28606 consensus_queue.cc:260] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:49.935102 28606 ts_tablet_manager.cc:1428] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260811 09:05:49.935307 28733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:49.934736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:56905)
0811 09:05:49.934955 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:49.935080 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:05:49.936595 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:49.936027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:60871)
0811 09:05:49.936232 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:49.936433 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 09:05:50.292980 28736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:50.293363 28736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:50.293556 28736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:50.301548 28736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:50.301753 28736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:50.301963 28736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:05:50.312532 28736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:35247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=38285
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:50.314046 28736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:50.315497 28736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:50.330387 28742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:50.330469 28741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:50.330497 28744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:50.334436 28736 server_base.cc:1034] running on GCE node
I20260811 09:05:50.335237 28736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:50.336395 28736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:50.337690 28736 hybrid_clock.cc:648] HybridClock initialized: now 1786439150337633 us; error 52 us; skew 500 ppm
I20260811 09:05:50.338092 28736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:50 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439150, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:50.341909 28736 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:05:50.343686 28736 webserver.cc:460] Webserver started at http://127.9.40.66:38285/ using document root <none> and password file <none>
I20260811 09:05:50.344420 28736 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:50.344537 28736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:50.350342 28736 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 09:05:50.353412 28754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.354861 28736 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:05:50.355058 28736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "8b3e3c70f6ef42798e54495927f3d6ea"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "6463d91a2ba9b6fb83061b65087e3957"
server_key_iv: "fa10c033d24498ef07be0ab5ecc50950"
server_key_version: "encryptionkey@0"
I20260811 09:05:50.356106 28736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:50.387158 28736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:50.391623 28736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:50.391938 28736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:50.393146 28736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:50.394662 28736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:50.394752 28736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.394918 28736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:50.395004 28736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.432631 28736 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35247
I20260811 09:05:50.432750 28888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35247 every 8 connection(s)
I20260811 09:05:50.434309 28736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:05:50.440254 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.437944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:45279)
0811 09:05:50.438871 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.439124 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":392,"threads_started":1}
W20260811 09:05:50.441097 28889 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:38941 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:38941: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
I20260811 09:05:50.442060 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.441709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:33591)
0811 09:05:50.441866 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.441941 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:05:50.442950 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.442631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:49797)
0811 09:05:50.442767 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.442848 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:05:50.443207 28889 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:05:50.444229 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28736
I20260811 09:05:50.444972 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:34591
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:38941
--webserver_port=38655
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:05:50.815052 28892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:05:50.815366 28892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:05:50.815557 28892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:05:50.823290 28892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:05:50.823421 28892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:05:50.823669 28892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:05:50.834439 28892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:34591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=38655
--tserver_master_addrs=127.9.40.126:38941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:05:50.835861 28892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:05:50.837397 28892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:05:50.851627 28900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:50.851691 28899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:05:50.852615 28902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:05:50.854017 28892 server_base.cc:1034] running on GCE node
I20260811 09:05:50.854738 28892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:05:50.855858 28892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:05:50.857156 28892 hybrid_clock.cc:648] HybridClock initialized: now 1786439150857091 us; error 57 us; skew 500 ppm
I20260811 09:05:50.857527 28892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:05:50 dist-test-slave-cbsf krb5kdc[27188](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439150, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:05:50.861157 28892 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:05:50.863010 28892 webserver.cc:460] Webserver started at http://127.9.40.67:38655/ using document root <none> and password file <none>
I20260811 09:05:50.863817 28892 fs_manager.cc:359] Metadata directory not provided
I20260811 09:05:50.863998 28892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:05:50.870277 28892 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 09:05:50.873531 28910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.874953 28892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:05:50.875161 28892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "dacf6dfe62e94e4cbfad31e775a84dff"
format_stamp: "Formatted at 2026-08-11 09:05:04 on dist-test-slave-cbsf"
server_key: "29d78c049df4176dfcf8ffe9a0ac2319"
server_key_iv: "4355c35c3a9fdf07d88df066d7e3fb48"
server_key_version: "encryptionkey@0"
I20260811 09:05:50.876238 28892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:05:50.891873 28892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:05:50.896301 28892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:05:50.896543 28892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:05:50.897663 28892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:05:50.899081 28892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:05:50.899176 28892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.899259 28892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:05:50.899317 28892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:05:50.937777 28892 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:34591
I20260811 09:05:50.937837 29039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:34591 every 8 connection(s)
I20260811 09:05:50.939508 28892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:05:50.940734 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.937786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:52319)
0811 09:05:50.940233 (+ 2447us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.940376 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2105,"thread_start_us":1963,"threads_started":1}
I20260811 09:05:50.940786 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 28892
I20260811 09:05:50.946314 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.943913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:47433)
0811 09:05:50.944798 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.945030 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
W20260811 09:05:50.947033 29044 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:38941 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:38941: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
I20260811 09:05:50.947882 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.947564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:36017)
0811 09:05:50.947703 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.947780 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:05:50.948895 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:50.948505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:57487)
0811 09:05:50.948646 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:50.948776 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:05:50.949179 29044 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:05:51.444567 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.443771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:40395)
0811 09:05:51.444222 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:51.444315 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":217,"threads_started":1}
I20260811 09:05:51.720113 29050 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:05:51.720456 29050 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:51.722335 29050 leader_election.cc:288] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247), dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
Aug 11 09:05:51 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439149, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
Aug 11 09:05:51 dist-test-slave-cbsf krb5kdc[27188](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439149, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:05:51.770117 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.723100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40051 (local address 127.9.40.66:35247)
0811 09:05:51.737016 (+ 13916us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:51.737034 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:51.737596 (+ 562us) server_negotiation.cc:407] Connection header received
0811 09:05:51.740974 (+ 3378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:51.740984 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:51.741518 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:51.741941 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:51.745203 (+ 3262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.747184 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.750307 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.751404 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.751605 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:51.761465 (+ 9860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:51.761480 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:51.761504 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:51.761562 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:51.765128 (+ 3566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:51.766177 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:51.766188 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:51.766196 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:51.766442 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:51.767002 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:51.767007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:51.767010 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:51.767830 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:51.768042 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:51.769214 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0811 09:05:51.769672 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13696,"thread_start_us":13475,"threads_started":1}
I20260811 09:05:51.770113 29054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.722991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:54009 (local address 127.9.40.67:34591)
0811 09:05:51.736467 (+ 13476us) server_negotiation.cc:206] Beginning negotiation
0811 09:05:51.736494 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 09:05:51.737679 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 09:05:51.740908 (+ 3229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:05:51.740926 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:05:51.741518 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:05:51.741941 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:05:51.745204 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.747184 (+ 1980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.750307 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.751404 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.751627 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:51.762042 (+ 10415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:05:51.762051 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:05:51.762063 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:05:51.762105 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:05:51.765382 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:51.766435 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:51.766445 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:51.766452 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:51.766729 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:05:51.767228 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:05:51.767233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:05:51.767236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:05:51.767830 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:05:51.768042 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:05:51.769212 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0811 09:05:51.769672 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13187,"thread_start_us":13036,"threads_started":1}
I20260811 09:05:51.770124 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.722669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35247 (local address 127.9.40.65:40051)
0811 09:05:51.737333 (+ 14664us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:51.737414 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:51.738128 (+ 714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:51.742165 (+ 4037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:51.742177 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:51.743563 (+ 1386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:51.744923 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:51.744955 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.747464 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.747469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:51.750122 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:51.750140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.751683 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.751688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:51.752100 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:51.757577 (+ 5477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:51.757624 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:51.761034 (+ 3410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:51.765304 (+ 4270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:51.765315 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:51.765330 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:51.765963 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:51.766583 (+ 620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:51.766589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:51.766593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:51.766856 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:51.768008 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:51.768017 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:51.768528 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 09:05:51.768901 (+ 373us) client_negotiation.cc:240] Negotiation successful
0811 09:05:51.769659 (+ 758us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14504,"mutex_wait_us":822,"spinlock_wait_cycles":126592,"thread_start_us":14554,"threads_started":1}
I20260811 09:05:51.770144 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.722725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34591 (local address 127.9.40.65:54009)
0811 09:05:51.737543 (+ 14818us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:51.737562 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:05:51.738128 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:05:51.742165 (+ 4037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:05:51.742177 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:05:51.743563 (+ 1386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:05:51.744923 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:51.744955 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.747463 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.747469 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:51.750122 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:05:51.750140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:05:51.751672 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:05:51.751679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:05:51.752097 (+ 418us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:05:51.757577 (+ 5480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:05:51.757621 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:05:51.761716 (+ 4095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:05:51.765623 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:51.765627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:51.765629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:51.766256 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:51.766877 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:05:51.766883 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:05:51.766889 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:05:51.767082 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:05:51.768008 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:05:51.768017 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:05:51.768528 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 09:05:51.768901 (+ 373us) client_negotiation.cc:240] Negotiation successful
0811 09:05:51.769536 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14691,"mutex_wait_us":505,"spinlock_wait_cycles":65792,"thread_start_us":14285,"threads_started":1}
W20260811 09:05:51.774258 28600 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): Not found: Tablet not found: 95795d17289147be851138c41328409f
W20260811 09:05:51.774686 28601 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): Not found: Tablet not found: 95795d17289147be851138c41328409f
I20260811 09:05:51.774873 28601 leader_election.cc:302] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56; no voters: 8b3e3c70f6ef42798e54495927f3d6ea, dacf6dfe62e94e4cbfad31e775a84dff
I20260811 09:05:51.775295 29050 raft_consensus.cc:2726] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:05:51.942349 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.941908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:38431)
0811 09:05:51.942095 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:51.942202 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 09:05:51.950659 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:51.949752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:43961)
0811 09:05:51.950268 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:51.950409 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":244,"threads_started":1}
I20260811 09:05:52.446409 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:52.445691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:55789)
0811 09:05:52.446149 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:52.446236 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":232,"threads_started":1}
I20260811 09:05:52.944144 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:52.943301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:33595)
0811 09:05:52.943866 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:52.943953 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":282,"threads_started":1}
I20260811 09:05:52.952591 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:52.951683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:36419)
0811 09:05:52.952235 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:52.952349 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":305,"threads_started":1}
I20260811 09:05:53.301306 29060 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:05:53.301524 29060 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:53.302280 29060 leader_election.cc:288] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247), dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
W20260811 09:05:53.303512 28601 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): Not found: Tablet not found: 95795d17289147be851138c41328409f
W20260811 09:05:53.303706 28600 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): Not found: Tablet not found: 95795d17289147be851138c41328409f
I20260811 09:05:53.303882 28600 leader_election.cc:302] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56; no voters: 8b3e3c70f6ef42798e54495927f3d6ea, dacf6dfe62e94e4cbfad31e775a84dff
I20260811 09:05:53.304172 29060 raft_consensus.cc:2726] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:05:53.448318 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:53.447555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:56921)
0811 09:05:53.448029 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:53.448136 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":247,"threads_started":1}
I20260811 09:05:53.946053 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:53.945289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:36935)
0811 09:05:53.945771 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:53.945863 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260811 09:05:53.954489 29063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:53.953729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:40849)
0811 09:05:53.954232 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:53.954319 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":266,"threads_started":1}
I20260811 09:05:54.450217 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:54.449445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:42807)
0811 09:05:54.449913 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:54.450012 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":232,"threads_started":1}
I20260811 09:05:54.948146 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:54.947199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:44865)
0811 09:05:54.947848 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:54.947941 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":404,"threads_started":1}
I20260811 09:05:54.956420 29066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:54.955621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:37733)
0811 09:05:54.956132 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:54.956222 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":255,"threads_started":1}
I20260811 09:05:55.452049 29067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:55.451243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:57827)
0811 09:05:55.451768 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:55.451862 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":254,"threads_started":1}
I20260811 09:05:55.949978 29069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:55.949243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:50159)
0811 09:05:55.949724 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:55.949813 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":273,"threads_started":1}
I20260811 09:05:55.958115 29070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:55.957427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:37491)
0811 09:05:55.957859 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:55.957939 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":226,"threads_started":1}
I20260811 09:05:56.122323 29071 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:05:56.122550 29071 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:05:56.123425 29071 leader_election.cc:288] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247), dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
W20260811 09:05:56.124579 28600 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): Not found: Tablet not found: 95795d17289147be851138c41328409f
W20260811 09:05:56.124832 28601 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): Not found: Tablet not found: 95795d17289147be851138c41328409f
I20260811 09:05:56.124967 28601 leader_election.cc:302] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56; no voters: 8b3e3c70f6ef42798e54495927f3d6ea, dacf6dfe62e94e4cbfad31e775a84dff
I20260811 09:05:56.125232 29071 raft_consensus.cc:2726] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:05:56.454391 29072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:56.453289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:38087)
0811 09:05:56.454025 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:56.454182 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":406,"threads_started":1}
I20260811 09:05:56.951870 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:56.951093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:35307)
0811 09:05:56.951578 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:56.951665 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":263,"threads_started":1}
I20260811 09:05:56.960036 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:56.959041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:42275)
0811 09:05:56.959618 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:56.959780 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":294,"threads_started":1}
I20260811 09:05:57.456202 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:57.455342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:46839)
0811 09:05:57.455887 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:57.455982 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":253,"threads_started":1}
I20260811 09:05:57.953768 29076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:57.953007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:56917)
0811 09:05:57.953516 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:57.953603 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":274,"threads_started":1}
I20260811 09:05:57.961838 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:57.961065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:57215)
0811 09:05:57.961558 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:57.961643 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":219,"threads_started":1}
I20260811 09:05:58.458207 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:58.457337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:52691)
0811 09:05:58.457911 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:58.458007 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":359,"threads_started":1}
I20260811 09:05:58.955513 29079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:58.954750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:44411)
0811 09:05:58.955232 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:58.955317 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":255,"threads_started":1}
I20260811 09:05:58.963917 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:58.962992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:55377)
0811 09:05:58.963529 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:58.963661 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":250,"threads_started":1}
I20260811 09:05:59.460083 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:59.459278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:40117)
0811 09:05:59.459795 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:59.459893 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":244,"threads_started":1}
I20260811 09:05:59.957702 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:59.956873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:51509)
0811 09:05:59.957414 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:59.957522 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":280,"threads_started":1}
I20260811 09:05:59.965828 29083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:05:59.965011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:43409)
0811 09:05:59.965570 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 09:05:59.965660 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":258,"threads_started":1}
I20260811 09:06:00.462081 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:00.461271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:45885)
0811 09:06:00.461786 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:00.461881 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":283,"threads_started":1}
I20260811 09:06:00.959476 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:00.958689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:47789)
0811 09:06:00.959179 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:00.959266 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":259,"threads_started":1}
I20260811 09:06:00.968014 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:00.967010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:34493)
0811 09:06:00.967673 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:00.967803 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":345,"threads_started":1}
I20260811 09:06:01.087817 29089 raft_consensus.cc:482] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:01.088071 29089 raft_consensus.cc:504] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3e3c70f6ef42798e54495927f3d6ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35247 } } peers { permanent_uuid: "dacf6dfe62e94e4cbfad31e775a84dff" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34591 } } peers { permanent_uuid: "4c7b9491adc64f22ba77984243fc4a56" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37599 } }
I20260811 09:06:01.088933 29089 leader_election.cc:288] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247), dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591)
W20260811 09:06:01.090016 28601 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8b3e3c70f6ef42798e54495927f3d6ea (127.9.40.66:35247): Not found: Tablet not found: 95795d17289147be851138c41328409f
W20260811 09:06:01.090211 28600 leader_election.cc:341] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dacf6dfe62e94e4cbfad31e775a84dff (127.9.40.67:34591): Not found: Tablet not found: 95795d17289147be851138c41328409f
I20260811 09:06:01.090320 28600 leader_election.cc:302] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c7b9491adc64f22ba77984243fc4a56; no voters: 8b3e3c70f6ef42798e54495927f3d6ea, dacf6dfe62e94e4cbfad31e775a84dff
I20260811 09:06:01.090557 29089 raft_consensus.cc:2726] T 95795d17289147be851138c41328409f P 4c7b9491adc64f22ba77984243fc4a56 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:06:01.463794 29090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:01.463025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:50791)
0811 09:06:01.463511 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:01.463614 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":229,"threads_started":1}
I20260811 09:06:01.961421 29091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:01.960552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:56209)
0811 09:06:01.961116 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:01.961216 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":357,"threads_started":1}
I20260811 09:06:01.969908 29093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:01.969021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:48755)
0811 09:06:01.969641 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:01.969735 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":352,"threads_started":1}
I20260811 09:06:02.465647 29094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:02.464883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:46989)
0811 09:06:02.465362 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:02.465450 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":235,"threads_started":1}
I20260811 09:06:02.963230 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:02.962452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:50597)
0811 09:06:02.962908 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:02.962995 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":250,"threads_started":1}
I20260811 09:06:02.971731 29096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:02.971019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:59281)
0811 09:06:02.971457 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:02.971535 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":224,"threads_started":1}
I20260811 09:06:03.467386 29097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:03.466611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:49743)
0811 09:06:03.467103 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:03.467206 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":235,"threads_started":1}
I20260811 09:06:03.965207 29098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:03.964346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:56201)
0811 09:06:03.964917 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:03.965008 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":277,"threads_started":1}
I20260811 09:06:03.973502 29099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:03.972797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:57129)
0811 09:06:03.973242 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:03.973334 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":216,"threads_started":1}
I20260811 09:06:04.469208 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:04.468315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:44783)
0811 09:06:04.468909 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:04.468998 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":372,"threads_started":1}
I20260811 09:06:04.967417 29101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:04.966540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:34313)
0811 09:06:04.967131 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:04.967225 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":326,"threads_started":1}
I20260811 09:06:04.975126 29102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:04.974372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:54695)
0811 09:06:04.974824 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:04.974902 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":231,"threads_started":1}
I20260811 09:06:05.471261 29103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:05.470379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.66:49729)
0811 09:06:05.470932 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:05.471042 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":284,"threads_started":1}
I20260811 09:06:05.946477 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 28529
I20260811 09:06:05.969485 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:05.968406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.65:49259)
0811 09:06:05.969092 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:05.969238 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":376,"threads_started":1}
I20260811 09:06:05.970834 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:05.948130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43458 (local address 127.9.40.65:37599)
0811 09:06:05.948758 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:05.948768 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:05.948812 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:06:05.948883 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:05.948890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:05.949102 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:05.949285 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:05.950852 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:05.951875 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:05.953305 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:05.953929 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:05.954086 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:05.963943 (+ 9857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:05.963962 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:05.963981 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:05.964026 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:05.967272 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:05.968214 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:05.968223 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:05.968226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:05.968420 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:05.968974 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:05.968979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:05.968981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:05.969462 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:05.969854 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:05.970194 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 09:06:05.970502 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":261,"threads_started":1}
I20260811 09:06:05.972149 28724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:05.977640 29108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:05.976504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38941 (local address 127.9.40.67:47243)
0811 09:06:05.977116 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:05.977348 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:38941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":343,"threads_started":1}
I20260811 09:06:06.067097 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 28529
I20260811 09:06:06.090323 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 28736
I20260811 09:06:06.105369 29109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:06.091216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47606 (local address 127.9.40.66:35247)
0811 09:06:06.091803 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:06.091807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:06.091825 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:06:06.091882 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:06.091886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:06.091957 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:06.092056 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:06.093579 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:06.094436 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:06.095669 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:06.096137 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:06.096269 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:06.099402 (+ 3133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:06.099408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:06.099412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:06.099453 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:06.102510 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:06.103309 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:06.103314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:06.103317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:06.103518 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:06.103933 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:06.103936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:06.103938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:06.104261 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:06.104514 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:06.104835 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 09:06:06.105055 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":358,"threads_started":1}
I20260811 09:06:06.106176 28884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:06.194394 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 28736
I20260811 09:06:06.218844 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 28892
I20260811 09:06:06.234499 29111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:06.219967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34376 (local address 127.9.40.67:34591)
0811 09:06:06.220647 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:06.220653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:06.220671 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:06:06.220778 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:06.220782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:06.220855 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:06.220954 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:06.222460 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:06.223293 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:06.224629 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:06.225204 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:06.225326 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:06.228253 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:06.228259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:06.228262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:06.228302 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:06.231377 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:06.232247 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:06.232252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:06.232254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:06.232409 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:06.232973 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:06.232977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:06.232980 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:06.233285 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:06.233505 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:06.233871 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 09:06:06.234059 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":367,"threads_started":1}
I20260811 09:06:06.235436 29035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:06.325071 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 28892
2026-08-11T09:06:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65405 ms)
[----------] 30 tests from SecurityITest (313813 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:06 dist-test-slave-cbsf krb5kdc[29115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:06 dist-test-slave-cbsf krb5kdc[29115](info): set up 2 sockets
Aug 11 09:06:06 dist-test-slave-cbsf krb5kdc[29115](info): commencing operation
krb5kdc: starting...
W20260811 09:06:07.576287 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:07 dist-test-slave-cbsf krb5kdc[29115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:06:07.924351 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42721
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:42721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:08.285954 29133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:08.286273 29133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:08.286402 29133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:08.294456 29133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:08.294577 29133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:08.294633 29133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:08.294693 29133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:08.294744 29133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:08.305043 29133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:42721
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:08.306397 29133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:08.307744 29133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:08.321486 29139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:08.321605 29138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:08.321539 29143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:08.322343 29133 server_base.cc:1034] running on GCE node
I20260811 09:06:08.323200 29133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:08.324743 29133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:08.325982 29133 hybrid_clock.cc:648] HybridClock initialized: now 1786439168325936 us; error 34 us; skew 500 ppm
I20260811 09:06:08.326385 29133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:08 dist-test-slave-cbsf krb5kdc[29115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439168, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:08.329793 29133 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:06:08.331372 29133 webserver.cc:460] Webserver started at http://127.9.40.126:46283/ using document root <none> and password file <none>
I20260811 09:06:08.332126 29133 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:08.332291 29133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:08.332638 29133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:08.335850 29133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1"
format_stamp: "Formatted at 2026-08-11 09:06:08 on dist-test-slave-cbsf"
server_key: "943288af53b5e16e5545a8e6e9899e8b"
server_key_iv: "64862f9c14ba6ca5f6e6e0eacdaa0d4d"
server_key_version: "encryptionkey@0"
I20260811 09:06:08.336673 29133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1"
format_stamp: "Formatted at 2026-08-11 09:06:08 on dist-test-slave-cbsf"
server_key: "943288af53b5e16e5545a8e6e9899e8b"
server_key_iv: "64862f9c14ba6ca5f6e6e0eacdaa0d4d"
server_key_version: "encryptionkey@0"
I20260811 09:06:08.342242 29133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:06:08.345911 29150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:08.347296 29133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:06:08.347500 29133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1"
format_stamp: "Formatted at 2026-08-11 09:06:08 on dist-test-slave-cbsf"
server_key: "943288af53b5e16e5545a8e6e9899e8b"
server_key_iv: "64862f9c14ba6ca5f6e6e0eacdaa0d4d"
server_key_version: "encryptionkey@0"
I20260811 09:06:08.347729 29133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:08.370247 29133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:08.374706 29133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:08.375037 29133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:08.395030 29133 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:42721
I20260811 09:06:08.395067 29208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:42721 every 8 connection(s)
I20260811 09:06:08.396538 29133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:08.398922 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29133
I20260811 09:06:08.399199 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:08.400071 9377 external_mini_cluster.cc:1428] Setting key be18a285799fcb447f6f82ccc3a3b4a1
I20260811 09:06:08.401459 29209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:06:08 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:08.417176 29209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: Bootstrap starting.
I20260811 09:06:08.421964 29209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:08.423363 29209 log.cc:824] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:08.427251 29209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: No bootstrap required, opened a new log
I20260811 09:06:08.427380 29212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:08.402356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55584 (local address 127.9.40.126:42721)
0811 09:06:08.403235 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:08.403247 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:08.403291 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:06:08.404975 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:08.404999 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:08.405412 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:08.405757 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:08.407641 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:08.409137 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:08.411111 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:08.411900 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:08.412063 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:08.417464 (+ 5401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:08.417476 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:08.417500 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:08.417570 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:08.421973 (+ 4403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:08.423048 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:08.423061 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:08.423074 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:08.423327 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:08.423819 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:08.423824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:08.423827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:08.424575 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:08.424921 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:08.425419 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 09:06:08.425895 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":408,"threads_started":1}
I20260811 09:06:08.435935 29209 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } }
I20260811 09:06:08.436420 29209 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:08.436583 29209 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccfb5676663e4e1ca2213f9ff6e6e1d1, State: Initialized, Role: FOLLOWER
I20260811 09:06:08.437335 29209 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } }
I20260811 09:06:08.437613 29209 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:08.437767 29209 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:08.437947 29209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:08.440263 29209 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } }
I20260811 09:06:08.440878 29209 leader_election.cc:302] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccfb5676663e4e1ca2213f9ff6e6e1d1; no voters:
I20260811 09:06:08.441620 29209 leader_election.cc:288] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:08.441777 29215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:08.442505 29215 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [term 1 LEADER]: Becoming Leader. State: Replica: ccfb5676663e4e1ca2213f9ff6e6e1d1, State: Running, Role: LEADER
I20260811 09:06:08.443162 29215 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } }
I20260811 09:06:08.444101 29209 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:08.446316 29217 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccfb5676663e4e1ca2213f9ff6e6e1d1. Latest consensus state: current_term: 1 leader_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } } }
I20260811 09:06:08.446291 29216 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccfb5676663e4e1ca2213f9ff6e6e1d1" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42721 } } }
I20260811 09:06:08.446856 29217 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:08.446895 29216 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:08.452287 29223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:08.458241 29223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:08.468216 29223 catalog_manager.cc:1347] Generated new cluster ID: 25fa98f415954ff8b4db1b69e53bd455
I20260811 09:06:08.468361 29223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:08.494077 29223 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:08.495504 29223 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:08.511461 29223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: Generated new TSK 0
I20260811 09:06:08.512431 29223 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:06:08.604854 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:08.969700 29241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:08.969998 29241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:08.970194 29241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:08.977874 29241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:08.978009 29241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:08.978214 29241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:08.988396 29241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:08.989856 29241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:08.991273 29241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:09.007513 29247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:09.007614 29248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:09.007614 29250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:09.008419 29241 server_base.cc:1034] running on GCE node
I20260811 09:06:09.009147 29241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:09.010145 29241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:09.011410 29241 hybrid_clock.cc:648] HybridClock initialized: now 1786439169011343 us; error 60 us; skew 500 ppm
I20260811 09:06:09.011837 29241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:09 dist-test-slave-cbsf krb5kdc[29115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439169, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:09.015182 29241 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:06:09.016739 29241 webserver.cc:460] Webserver started at http://127.9.40.65:44269/ using document root <none> and password file <none>
I20260811 09:06:09.017491 29241 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:09.017609 29241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:09.017890 29241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:09.021134 29241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "bd1c325aa11942ef9a9f0791a19dad3c"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "aa23245805345168a27dede6ba96ab6e"
server_key_iv: "5407655f9625a9e6d2a5ba784dbaf7a3"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.021987 29241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "bd1c325aa11942ef9a9f0791a19dad3c"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "aa23245805345168a27dede6ba96ab6e"
server_key_iv: "5407655f9625a9e6d2a5ba784dbaf7a3"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.027915 29241 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:06:09.031785 29257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.033315 29241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:09.033511 29241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "bd1c325aa11942ef9a9f0791a19dad3c"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "aa23245805345168a27dede6ba96ab6e"
server_key_iv: "5407655f9625a9e6d2a5ba784dbaf7a3"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.033746 29241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:09.045979 29241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:09.050621 29241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:09.051019 29241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:09.052259 29241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:09.053887 29241 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:09.053964 29241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.054154 29241 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:09.054219 29241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.090219 29241 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:39013
I20260811 09:06:09.090296 29383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:39013 every 8 connection(s)
I20260811 09:06:09.091665 29241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:09.101047 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29241
I20260811 09:06:09.101284 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:09.101965 9377 external_mini_cluster.cc:1428] Setting key 80090e722f1e7b428857c7cc90bc8144
Aug 11 09:06:09 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439169, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:09.114394 29386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:09.094454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58539 (local address 127.9.40.126:42721)
0811 09:06:09.094891 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:09.094896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:09.095927 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 09:06:09.097813 (+ 1886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:09.097817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:09.097886 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:09.097986 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:09.099549 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.100261 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.102084 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.102488 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.102651 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:09.106942 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:09.106952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:09.106959 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:09.107007 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:09.110769 (+ 3762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:09.111614 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:09.111622 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:09.111628 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:09.111784 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:09.112210 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:09.112218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:09.112224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:09.112626 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:09.112808 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:09.113570 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 09:06:09.113839 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260811 09:06:09.115036 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:09.095007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42721 (local address 127.9.40.65:58539)
0811 09:06:09.095747 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:09.095793 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:09.097556 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:09.098211 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:09.098220 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:09.098778 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:09.099381 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:09.099399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.100410 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.100414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:09.101921 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:09.101933 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.102728 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.102734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:09.103044 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:09.103808 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:09.103844 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:09.106702 (+ 2858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:09.110935 (+ 4233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:09.110943 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:09.110958 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:09.111466 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:09.111893 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:09.111896 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:09.111898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:09.112091 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:09.112797 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:09.112804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:09.113162 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 09:06:09.113401 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 09:06:09.113832 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":299,"threads_started":1}
I20260811 09:06:09.116811 29384 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42721
I20260811 09:06:09.117190 29384 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:09.118008 29384 heartbeater.cc:511] Master 127.9.40.126:42721 requested a full tablet report, sending...
I20260811 09:06:09.120647 29169 ts_manager.cc:194] Registered new tserver with Master: bd1c325aa11942ef9a9f0791a19dad3c (127.9.40.65:39013)
I20260811 09:06:09.122190 29169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:58539
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:06:09.181901 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:09.547603 29390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:09.547967 29390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:09.548154 29390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:09.556143 29390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:09.556308 29390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:09.556504 29390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:09.568476 29390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:09.569940 29390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:09.571379 29390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:09.585703 29397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:09.585747 29399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:09.585661 29396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:09.586690 29390 server_base.cc:1034] running on GCE node
I20260811 09:06:09.587369 29390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:09.588399 29390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:09.589694 29390 hybrid_clock.cc:648] HybridClock initialized: now 1786439169589610 us; error 92 us; skew 500 ppm
I20260811 09:06:09.590099 29390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:09 dist-test-slave-cbsf krb5kdc[29115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439169, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:09.593454 29390 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:06:09.595093 29390 webserver.cc:460] Webserver started at http://127.9.40.66:37343/ using document root <none> and password file <none>
I20260811 09:06:09.595863 29390 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:09.595968 29390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:09.596244 29390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:09.599390 29390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "b7dbd87302e7490c90df2c3987d6493e"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "b13ae548e50d0ffab3198098154df8ad"
server_key_iv: "f4ae7c576fa1d8ec6ff56c22359ae914"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.600199 29390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "b7dbd87302e7490c90df2c3987d6493e"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "b13ae548e50d0ffab3198098154df8ad"
server_key_iv: "f4ae7c576fa1d8ec6ff56c22359ae914"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.605996 29390 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 09:06:09.609575 29407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.610944 29390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:09.611137 29390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "b7dbd87302e7490c90df2c3987d6493e"
format_stamp: "Formatted at 2026-08-11 09:06:09 on dist-test-slave-cbsf"
server_key: "b13ae548e50d0ffab3198098154df8ad"
server_key_iv: "f4ae7c576fa1d8ec6ff56c22359ae914"
server_key_version: "encryptionkey@0"
I20260811 09:06:09.611359 29390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:09.631126 29390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:09.635659 29390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:09.636061 29390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:09.637284 29390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:09.638906 29390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:09.639031 29390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.639197 29390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:09.639250 29390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:09.675638 29390 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:40507
I20260811 09:06:09.675685 29544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:40507 every 8 connection(s)
I20260811 09:06:09.677071 29390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:09.687332 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29390
I20260811 09:06:09.687575 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:09.688052 9377 external_mini_cluster.cc:1428] Setting key 9b10cf62cf2725d09933aab23f67d287
Aug 11 09:06:09 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439169, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:09.697919 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:09.679732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42193 (local address 127.9.40.126:42721)
0811 09:06:09.680188 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:09.680193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:09.681247 (+ 1054us) server_negotiation.cc:407] Connection header received
0811 09:06:09.683137 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:09.683143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:09.683239 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:09.683366 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:09.684978 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.685736 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.687336 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.687895 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.688030 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:09.691861 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:09.691866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:09.691869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:09.691903 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:09.694592 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:09.695370 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:09.695382 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:09.695386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:09.695527 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:09.696002 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:09.696006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:09.696009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:09.696280 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:09.696454 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:09.697254 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 09:06:09.697517 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":228,"threads_started":1}
I20260811 09:06:09.698947 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:09.680286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42721 (local address 127.9.40.66:42193)
0811 09:06:09.681079 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:09.681125 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:09.682856 (+ 1731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:09.683615 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:09.683623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:09.684169 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:09.684804 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:09.684825 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.685904 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.685914 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:09.687191 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:09.687203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:09.688157 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:09.688165 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:09.688399 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:09.689189 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:09.689218 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:09.691624 (+ 2406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:09.694740 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:09.694750 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:09.694769 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:09.695234 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:09.695655 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:09.695660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:09.695664 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:09.695872 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:09.696416 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:09.696427 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:09.696857 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 09:06:09.697159 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 09:06:09.697745 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":304,"threads_started":1}
I20260811 09:06:09.700624 29545 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42721
I20260811 09:06:09.700995 29545 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:09.701781 29545 heartbeater.cc:511] Master 127.9.40.126:42721 requested a full tablet report, sending...
I20260811 09:06:09.704066 29169 ts_manager.cc:194] Registered new tserver with Master: b7dbd87302e7490c90df2c3987d6493e (127.9.40.66:40507)
I20260811 09:06:09.705351 29169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:42193
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:06:09.764137 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:06:10.125550 29384 heartbeater.cc:503] Master 127.9.40.126:42721 was elected leader, sending a full tablet report...
W20260811 09:06:10.132097 29551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:10.132408 29551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:10.132583 29551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:10.140261 29551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:10.140390 29551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:10.140609 29551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:10.150990 29551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42721
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:10.152626 29551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:10.154172 29551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:10.168869 29557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:10.168993 29556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:10.168954 29559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:10.169849 29551 server_base.cc:1034] running on GCE node
I20260811 09:06:10.170457 29551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:10.171669 29551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:10.172870 29551 hybrid_clock.cc:648] HybridClock initialized: now 1786439170172800 us; error 62 us; skew 500 ppm
I20260811 09:06:10.173249 29551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:10 dist-test-slave-cbsf krb5kdc[29115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439170, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:10.176578 29551 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:06:10.178218 29551 webserver.cc:460] Webserver started at http://127.9.40.67:39479/ using document root <none> and password file <none>
I20260811 09:06:10.179065 29551 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:10.179175 29551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:10.179456 29551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:10.182513 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "780f50d6f3674b9094d719ab4280c900"
format_stamp: "Formatted at 2026-08-11 09:06:10 on dist-test-slave-cbsf"
server_key: "d2bac77a9e5b546ef60167bc9f3e5615"
server_key_iv: "56dfe50556aa4b48f95d2e0591f5118f"
server_key_version: "encryptionkey@0"
I20260811 09:06:10.183306 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "780f50d6f3674b9094d719ab4280c900"
format_stamp: "Formatted at 2026-08-11 09:06:10 on dist-test-slave-cbsf"
server_key: "d2bac77a9e5b546ef60167bc9f3e5615"
server_key_iv: "56dfe50556aa4b48f95d2e0591f5118f"
server_key_version: "encryptionkey@0"
I20260811 09:06:10.188784 29551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 09:06:10.192239 29569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:10.193620 29551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:06:10.193786 29551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "780f50d6f3674b9094d719ab4280c900"
format_stamp: "Formatted at 2026-08-11 09:06:10 on dist-test-slave-cbsf"
server_key: "d2bac77a9e5b546ef60167bc9f3e5615"
server_key_iv: "56dfe50556aa4b48f95d2e0591f5118f"
server_key_version: "encryptionkey@0"
I20260811 09:06:10.194005 29551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:10.210949 29551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:10.215467 29551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:10.215782 29551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:10.216961 29551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:10.218530 29551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:10.218618 29551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:10.218787 29551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:10.218842 29551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:10.255965 29551 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:43107
I20260811 09:06:10.256079 29694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:43107 every 8 connection(s)
I20260811 09:06:10.257402 29551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:10.259547 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29551
I20260811 09:06:10.259783 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:10.260291 9377 external_mini_cluster.cc:1428] Setting key f890ed50b4717e44dc2b4d96b5147c3f
Aug 11 09:06:10 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439170, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:10.283331 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.260376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:32917 (local address 127.9.40.126:42721)
0811 09:06:10.260940 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.260947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.262945 (+ 1998us) server_negotiation.cc:407] Connection header received
0811 09:06:10.264991 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.264998 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.265090 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.265214 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:10.267631 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.268934 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.272085 (+ 3151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.272696 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.272852 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.276670 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:10.276682 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:10.276687 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:10.276767 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:10.280042 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.280841 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.280848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.280852 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.280993 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.281448 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.281454 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.281457 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.281765 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:10.281921 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.282694 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.282935 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":279,"threads_started":1}
I20260811 09:06:10.284319 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.260998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42721 (local address 127.9.40.67:32917)
0811 09:06:10.261905 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.261970 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.264650 (+ 2680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.265526 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.265539 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.266453 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:10.267381 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.267412 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.269200 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.269206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.271902 (+ 2696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.271920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.272992 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.273001 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.273373 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.274116 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:10.274142 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:10.276428 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:10.280206 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:10.280216 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:10.280234 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:10.280686 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:10.281125 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:10.281130 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:10.281134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:10.281335 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:10.281910 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:10.281919 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:10.282305 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.282589 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.283166 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":328,"threads_started":1}
I20260811 09:06:10.285825 29695 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42721
I20260811 09:06:10.286149 29695 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:10.286867 29695 heartbeater.cc:511] Master 127.9.40.126:42721 requested a full tablet report, sending...
I20260811 09:06:10.288375 29169 ts_manager.cc:194] Registered new tserver with Master: 780f50d6f3674b9094d719ab4280c900 (127.9.40.67:43107)
I20260811 09:06:10.289377 29169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:32917
I20260811 09:06:10.291363 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:10.311329 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.296627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55596 (local address 127.9.40.126:42721)
0811 09:06:10.296852 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.296857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.297334 (+ 477us) server_negotiation.cc:407] Connection header received
0811 09:06:10.297474 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.297478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.297549 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.297682 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:10.299570 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.300412 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.302055 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.302514 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.302675 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.305526 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:10.305532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:10.305536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:10.305576 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:10.308367 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.309200 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.309205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.309210 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.309345 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.309755 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.309759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.309761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.310105 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:10.310382 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.310751 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.310992 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 09:06:10.315954 29168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:10.323381 29168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:10.352560 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.336660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:43107 (local address 127.0.0.1:52764)
0811 09:06:10.338990 (+ 2330us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.339045 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.339254 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.340529 (+ 1275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.340542 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.340584 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:10.341174 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.341193 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.343991 (+ 2798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.344000 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.347562 (+ 3562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.347591 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.351149 (+ 3558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.351165 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.351534 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.351627 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.351854 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.352132 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2076,"thread_start_us":1902,"threads_started":1}
I20260811 09:06:10.354143 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.337048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52764 (local address 127.9.40.67:43107)
0811 09:06:10.338559 (+ 1511us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.338571 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.339461 (+ 890us) server_negotiation.cc:407] Connection header received
0811 09:06:10.339591 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.339603 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.339918 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.340122 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:10.341449 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.343764 (+ 2315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.347869 (+ 4105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.350061 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.350258 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.350624 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.353449 (+ 2825us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.353728 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1252,"thread_start_us":1097,"threads_started":1}
I20260811 09:06:10.358129 29622 tablet_service.cc:1467] Processing CreateTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 (DEFAULT_TABLE table=test-table [id=7d1f387895b04d37871c2990ac17a7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:10.359758 29622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ed9d2a53f84a5190f60fcdc81bac81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:10.361166 29714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.337153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:40507 (local address 127.0.0.1:43646)
0811 09:06:10.351985 (+ 14832us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.352009 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.352193 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.353351 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.353359 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.353409 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:10.353944 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.353955 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.356193 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.356199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.358724 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.358734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.360116 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.360124 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.360307 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.360349 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.360476 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.360632 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14562,"mutex_wait_us":243,"spinlock_wait_cycles":2432,"thread_start_us":13641,"threads_started":1}
I20260811 09:06:10.361547 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.337875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.9.40.66:40507)
0811 09:06:10.349667 (+ 11792us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.349680 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.352351 (+ 2671us) server_negotiation.cc:407] Connection header received
0811 09:06:10.352644 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.352651 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.352864 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.353085 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:10.354178 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.356011 (+ 1833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.358912 (+ 2901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.359918 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.360057 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.360292 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.360998 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.361244 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11603,"thread_start_us":11475,"threads_started":1}
I20260811 09:06:10.362805 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.339370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57082)
0811 09:06:10.352080 (+ 12710us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.352095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.352202 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.353355 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.353360 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.353399 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:10.354084 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.354099 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.356193 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.356199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.358692 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.358711 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.361880 (+ 3169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.361889 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.362107 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.362149 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.362232 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.362372 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12584,"mutex_wait_us":51,"spinlock_wait_cycles":10752,"thread_start_us":12301,"threads_started":1}
I20260811 09:06:10.363890 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.339793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.9.40.65:39013)
0811 09:06:10.351461 (+ 11668us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.351469 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.352349 (+ 880us) server_negotiation.cc:407] Connection header received
0811 09:06:10.352479 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.352488 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.352870 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.353085 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:10.354257 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.356011 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.358908 (+ 2897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.360744 (+ 1836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.360881 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.363282 (+ 2401us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.363361 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.363542 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11451,"thread_start_us":11530,"threads_started":1}
I20260811 09:06:10.366109 29468 tablet_service.cc:1467] Processing CreateTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 (DEFAULT_TABLE table=test-table [id=7d1f387895b04d37871c2990ac17a7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:10.367806 29468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ed9d2a53f84a5190f60fcdc81bac81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:10.367564 29311 tablet_service.cc:1467] Processing CreateTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 (DEFAULT_TABLE table=test-table [id=7d1f387895b04d37871c2990ac17a7de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:10.369082 29311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ed9d2a53f84a5190f60fcdc81bac81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:10.374872 29718 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Bootstrap starting.
I20260811 09:06:10.378239 29718 tablet_bootstrap.cc:654] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:10.379496 29718 log.cc:824] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:10.383003 29718 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: No bootstrap required, opened a new log
I20260811 09:06:10.383417 29718 ts_tablet_manager.cc:1397] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260811 09:06:10.383395 29720 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Bootstrap starting.
I20260811 09:06:10.387097 29720 tablet_bootstrap.cc:654] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:10.387967 29721 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Bootstrap starting.
I20260811 09:06:10.388824 29720 log.cc:824] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:10.391214 29721 tablet_bootstrap.cc:654] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:10.392529 29721 log.cc:824] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:10.394424 29720 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: No bootstrap required, opened a new log
I20260811 09:06:10.394724 29720 ts_tablet_manager.cc:1397] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260811 09:06:10.394548 29718 raft_consensus.cc:348] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.395116 29718 raft_consensus.cc:374] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:10.395223 29718 raft_consensus.cc:729] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 780f50d6f3674b9094d719ab4280c900, State: Initialized, Role: FOLLOWER
I20260811 09:06:10.395888 29721 tablet_bootstrap.cc:492] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: No bootstrap required, opened a new log
I20260811 09:06:10.396119 29721 ts_tablet_manager.cc:1397] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 09:06:10.395870 29718 consensus_queue.cc:260] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.407338 29721 raft_consensus.cc:348] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.407857 29721 raft_consensus.cc:374] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:10.407982 29721 raft_consensus.cc:729] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd1c325aa11942ef9a9f0791a19dad3c, State: Initialized, Role: FOLLOWER
I20260811 09:06:10.408802 29721 consensus_queue.cc:260] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.411486 29720 raft_consensus.cc:348] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.412101 29720 raft_consensus.cc:374] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:10.412235 29720 raft_consensus.cc:729] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7dbd87302e7490c90df2c3987d6493e, State: Initialized, Role: FOLLOWER
I20260811 09:06:10.415285 29695 heartbeater.cc:503] Master 127.9.40.126:42721 was elected leader, sending a full tablet report...
I20260811 09:06:10.415997 29718 ts_tablet_manager.cc:1428] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Time spent starting tablet: real 0.032s user 0.017s sys 0.014s
I20260811 09:06:10.416942 29721 ts_tablet_manager.cc:1428] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Time spent starting tablet: real 0.021s user 0.016s sys 0.004s
I20260811 09:06:10.417019 29720 consensus_queue.cc:260] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.421241 29545 heartbeater.cc:503] Master 127.9.40.126:42721 was elected leader, sending a full tablet report...
I20260811 09:06:10.421927 29720 ts_tablet_manager.cc:1428] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Time spent starting tablet: real 0.027s user 0.020s sys 0.002s
W20260811 09:06:10.429226 29546 tablet.cc:2369] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:10.507231 29725 raft_consensus.cc:482] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:10.507529 29725 raft_consensus.cc:504] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
W20260811 09:06:10.509186 29696 tablet.cc:2369] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:10.509560 29725 leader_election.cc:288] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 780f50d6f3674b9094d719ab4280c900 (127.9.40.67:43107), b7dbd87302e7490c90df2c3987d6493e (127.9.40.66:40507)
I20260811 09:06:10.517745 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.510428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53601 (local address 127.9.40.66:40507)
0811 09:06:10.510755 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.510766 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.511488 (+ 722us) server_negotiation.cc:407] Connection header received
0811 09:06:10.511638 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.511648 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.511758 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.511879 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:10.512534 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.513569 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.515637 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.516649 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.516820 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.516919 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.517308 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.517451 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 09:06:10.517746 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.510451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:49581 (local address 127.9.40.67:43107)
0811 09:06:10.510726 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.510736 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.511729 (+ 993us) server_negotiation.cc:407] Connection header received
0811 09:06:10.511924 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.511929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.512010 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.512128 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:10.512994 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.513797 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.515936 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.516826 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.516958 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.517064 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.517375 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.517511 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:06:10.517859 29729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.510078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:40507 (local address 127.9.40.65:53601)
0811 09:06:10.511355 (+ 1277us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.511371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.511505 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.511909 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.511914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.511940 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:10.512357 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.512369 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.513711 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.513716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.515477 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.515489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.516924 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.516931 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.517126 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.517163 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.517240 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.517375 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1151,"mutex_wait_us":21,"thread_start_us":950,"threads_started":1}
I20260811 09:06:10.518550 29728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.510122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:43107 (local address 127.9.40.65:49581)
0811 09:06:10.511602 (+ 1480us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.511623 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:10.511774 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:10.512184 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:10.512188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:10.512219 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:10.512789 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.512804 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.514037 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.514041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.515779 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:10.515796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.517052 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.517057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:10.517188 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.517210 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:06:10.517331 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 09:06:10.517452 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1320,"mutex_wait_us":22,"thread_start_us":853,"threads_started":1}
I20260811 09:06:10.519043 29491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ed9d2a53f84a5190f60fcdc81bac81" candidate_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7dbd87302e7490c90df2c3987d6493e" is_pre_election: true
I20260811 09:06:10.519583 29491 raft_consensus.cc:2454] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd1c325aa11942ef9a9f0791a19dad3c in term 0.
I20260811 09:06:10.519551 29645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ed9d2a53f84a5190f60fcdc81bac81" candidate_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "780f50d6f3674b9094d719ab4280c900" is_pre_election: true
I20260811 09:06:10.520062 29645 raft_consensus.cc:2454] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd1c325aa11942ef9a9f0791a19dad3c in term 0.
I20260811 09:06:10.520300 29260 leader_election.cc:302] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b7dbd87302e7490c90df2c3987d6493e, bd1c325aa11942ef9a9f0791a19dad3c; no voters:
I20260811 09:06:10.520751 29725 raft_consensus.cc:2781] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:10.520920 29725 raft_consensus.cc:482] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:10.521039 29725 raft_consensus.cc:3037] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:10.523298 29725 raft_consensus.cc:504] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.523958 29725 leader_election.cc:288] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [CANDIDATE]: Term 1 election: Requested vote from peers 780f50d6f3674b9094d719ab4280c900 (127.9.40.67:43107), b7dbd87302e7490c90df2c3987d6493e (127.9.40.66:40507)
I20260811 09:06:10.524456 29645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ed9d2a53f84a5190f60fcdc81bac81" candidate_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "780f50d6f3674b9094d719ab4280c900"
I20260811 09:06:10.524518 29491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ed9d2a53f84a5190f60fcdc81bac81" candidate_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7dbd87302e7490c90df2c3987d6493e"
I20260811 09:06:10.524699 29645 raft_consensus.cc:3037] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:10.524801 29491 raft_consensus.cc:3037] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:10.527086 29645 raft_consensus.cc:2454] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd1c325aa11942ef9a9f0791a19dad3c in term 1.
I20260811 09:06:10.527077 29491 raft_consensus.cc:2454] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd1c325aa11942ef9a9f0791a19dad3c in term 1.
I20260811 09:06:10.527596 29259 leader_election.cc:302] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 780f50d6f3674b9094d719ab4280c900, bd1c325aa11942ef9a9f0791a19dad3c; no voters:
I20260811 09:06:10.527977 29725 raft_consensus.cc:2781] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:10.528467 29725 raft_consensus.cc:686] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 1 LEADER]: Becoming Leader. State: Replica: bd1c325aa11942ef9a9f0791a19dad3c, State: Running, Role: LEADER
I20260811 09:06:10.529048 29725 consensus_queue.cc:237] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } }
I20260811 09:06:10.534130 29168 catalog_manager.cc:5495] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c reported cstate change: term changed from 0 to 1, leader changed from <none> to bd1c325aa11942ef9a9f0791a19dad3c (127.9.40.65). New cstate: current_term: 1 leader_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd1c325aa11942ef9a9f0791a19dad3c" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39013 } health_report { overall_health: HEALTHY } } }
I20260811 09:06:10.554277 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.548638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.9.40.65:39013)
0811 09:06:10.548865 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.548870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.548887 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:06:10.549065 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.549069 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.549153 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.549285 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:10.549663 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.550419 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.551911 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.552338 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.552469 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.553092 (+ 623us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:10.553665 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:10.553846 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.553942 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.554015 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:06:10.562898 29645 raft_consensus.cc:1261] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 1 FOLLOWER]: Refusing update from remote peer bd1c325aa11942ef9a9f0791a19dad3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:10.562934 29491 raft_consensus.cc:1261] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 1 FOLLOWER]: Refusing update from remote peer bd1c325aa11942ef9a9f0791a19dad3c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:10.563952 29731 consensus_queue.cc:1035] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "780f50d6f3674b9094d719ab4280c900" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:10.564538 29725 consensus_queue.cc:1035] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7dbd87302e7490c90df2c3987d6493e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 40507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:10.576922 29736 mvcc.cc:205] Tried to move back new op lower bound from 7317254842614976512 to 7317254842490773504. Current Snapshot: MvccSnapshot[applied={T|T < 7317254842614976512}]
I20260811 09:06:10.594316 29168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:10.594616 29168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:10.598626 29168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ccfb5676663e4e1ca2213f9ff6e6e1d1: Sending DeleteTablet for 3 replicas of tablet 14ed9d2a53f84a5190f60fcdc81bac81
I20260811 09:06:10.599901 29468 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:10 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:43646
I20260811 09:06:10.600214 29622 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:10 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:52764
I20260811 09:06:10.600992 29311 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ed9d2a53f84a5190f60fcdc81bac81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:10 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:57082
I20260811 09:06:10.601594 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29241
Aug 11 09:06:10 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:06:10.619683 29746 tablet_replica.cc:330] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: stopping tablet replica
I20260811 09:06:10.620486 29745 tablet_replica.cc:330] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: stopping tablet replica
I20260811 09:06:10.621235 29746 raft_consensus.cc:2229] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:10.621223 29745 raft_consensus.cc:2229] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:10.621948 29745 raft_consensus.cc:2258] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:10.623423 29746 raft_consensus.cc:2258] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:10.625393 29745 ts_tablet_manager.cc:1905] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:10.629135 29744 tablet_replica.cc:330] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: stopping tablet replica
I20260811 09:06:10.629719 29744 raft_consensus.cc:2229] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:10.630362 29744 raft_consensus.cc:2258] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:10.631827 29745 ts_tablet_manager.cc:1918] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:10.632084 29745 log.cc:1196] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/14ed9d2a53f84a5190f60fcdc81bac81
I20260811 09:06:10.632544 29746 ts_tablet_manager.cc:1905] T 14ed9d2a53f84a5190f60fcdc81bac81 P bd1c325aa11942ef9a9f0791a19dad3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:10.632807 29745 ts_tablet_manager.cc:1939] T 14ed9d2a53f84a5190f60fcdc81bac81 P 780f50d6f3674b9094d719ab4280c900: Deleting consensus metadata
I20260811 09:06:10.633605 29717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.602478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57090 (local address 127.9.40.65:39013)
0811 09:06:10.602677 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.602686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.604129 (+ 1443us) server_negotiation.cc:407] Connection header received
0811 09:06:10.604432 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.604438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.604540 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.604634 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:10.607133 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.608809 (+ 1676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.611593 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.612682 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.612859 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.622931 (+ 10072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:10.622948 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:10.622969 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:10.623030 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:10.627429 (+ 4399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.630131 (+ 2702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.630142 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.630146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.630346 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.631000 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.631008 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.631014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.631886 (+ 872us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:10.632210 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.632908 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.633207 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:06:10.634900 29152 catalog_manager.cc:4841] TS 780f50d6f3674b9094d719ab4280c900 (127.9.40.67:43107): tablet 14ed9d2a53f84a5190f60fcdc81bac81 (table test-table [id=7d1f387895b04d37871c2990ac17a7de]) successfully deleted
I20260811 09:06:10.635331 29744 ts_tablet_manager.cc:1905] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:10.635586 29379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:10.642033 29744 ts_tablet_manager.cc:1918] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:10.642246 29744 log.cc:1196] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/14ed9d2a53f84a5190f60fcdc81bac81
I20260811 09:06:10.642765 29744 ts_tablet_manager.cc:1939] T 14ed9d2a53f84a5190f60fcdc81bac81 P b7dbd87302e7490c90df2c3987d6493e: Deleting consensus metadata
I20260811 09:06:10.644587 29153 catalog_manager.cc:4841] TS b7dbd87302e7490c90df2c3987d6493e (127.9.40.66:40507): tablet 14ed9d2a53f84a5190f60fcdc81bac81 (table test-table [id=7d1f387895b04d37871c2990ac17a7de]) successfully deleted
I20260811 09:06:10.753662 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29241
W20260811 09:06:10.778635 29151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:39013 (error 108)
I20260811 09:06:10.778887 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29390
I20260811 09:06:10.780803 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.780292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57106)
0811 09:06:10.780461 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.780593 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:06:10.781069 29151 catalog_manager.cc:4584] TS bd1c325aa11942ef9a9f0791a19dad3c (127.9.40.65:39013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14ed9d2a53f84a5190f60fcdc81bac81: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Aug 11 09:06:10 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:06:10.795336 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.779815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43652 (local address 127.9.40.66:40507)
0811 09:06:10.780011 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.780020 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.780044 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:06:10.780289 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.780294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.780389 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.780532 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:10.782268 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.783051 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.784315 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.785039 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.785140 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.789446 (+ 4306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:10.789462 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:10.789478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:10.789513 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:10.792467 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.793250 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.793257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.793260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.793389 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.793772 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.793776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.793779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.794236 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:10.794536 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.794702 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.794954 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:06:10.796064 29540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:10.834743 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.834229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57122)
0811 09:06:10.834466 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.834574 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 09:06:10.835917 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.835521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57138)
0811 09:06:10.835713 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.835795 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 09:06:10.890565 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29390
I20260811 09:06:10.896914 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.896277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57144)
0811 09:06:10.896551 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.896675 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 09:06:10.898157 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.897679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57154)
0811 09:06:10.897877 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.897977 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:06:10.914094 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29551
Aug 11 09:06:10 dist-test-slave-cbsf krb5kdc[29115](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:06:10.930094 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.914929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52776 (local address 127.9.40.67:43107)
0811 09:06:10.915100 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:10.915105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:10.915124 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:06:10.915286 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:10.915291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:10.915360 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:10.915447 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:10.916873 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.917636 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.918885 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:10.919301 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:10.919407 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:10.924225 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:10.924244 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:10.924263 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:10.924309 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:10.927116 (+ 2807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.927851 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.927859 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.927862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.928015 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:10.928387 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:10.928391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:10.928393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:10.929063 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:10.929317 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:10.929611 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 09:06:10.929799 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:06:10.930833 29689 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:10.979246 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.978618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57162)
0811 09:06:10.978891 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.979008 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 09:06:10.980509 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:10.980058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57176)
0811 09:06:10.980282 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:10.980368 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 09:06:11.028477 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29551
I20260811 09:06:11.050663 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29133
I20260811 09:06:11.051998 29204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:11.169358 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29133
I20260811 09:06:11.169423 29716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:11.168851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39013 (local address 127.0.0.1:57184)
0811 09:06:11.169065 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:11.169242 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
2026-08-11T09:06:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4850 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T09:06:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:11Z Disabled control of system clock
I20260811 09:06:11.274739 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44157
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44157
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:11.633160 29758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:11.633497 29758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:11.633684 29758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:11.642009 29758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:11.642138 29758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:11.642241 29758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:11.642320 29758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:11.642408 29758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:11.652138 29758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44157
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:11.653611 29758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:11.655515 29758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:11.667675 29769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:11.667732 29766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:11.667755 29767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:11.668947 29758 server_base.cc:1034] running on GCE node
I20260811 09:06:11.669692 29758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:11.671109 29758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:11.672349 29758 hybrid_clock.cc:648] HybridClock initialized: now 1786439171672312 us; error 51 us; skew 500 ppm
I20260811 09:06:11.672842 29758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:11.681835 29758 webserver.cc:460] Webserver started at http://127.9.40.126:36847/ using document root <none> and password file <none>
I20260811 09:06:11.682593 29758 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:11.682746 29758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:11.683101 29758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:11.686232 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "b2b373cfd55f44719471a2c4a9299064"
format_stamp: "Formatted at 2026-08-11 09:06:11 on dist-test-slave-cbsf"
server_key: "b784c4286d4a86a0ffc3053bd7cb3170"
server_key_iv: "98dc589316150467ac65e05c42b4a1ce"
server_key_version: "encryptionkey@0"
I20260811 09:06:11.687125 29758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "b2b373cfd55f44719471a2c4a9299064"
format_stamp: "Formatted at 2026-08-11 09:06:11 on dist-test-slave-cbsf"
server_key: "b784c4286d4a86a0ffc3053bd7cb3170"
server_key_iv: "98dc589316150467ac65e05c42b4a1ce"
server_key_version: "encryptionkey@0"
I20260811 09:06:11.692413 29758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:06:11.695981 29776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:11.697395 29758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:11.697607 29758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "b2b373cfd55f44719471a2c4a9299064"
format_stamp: "Formatted at 2026-08-11 09:06:11 on dist-test-slave-cbsf"
server_key: "b784c4286d4a86a0ffc3053bd7cb3170"
server_key_iv: "98dc589316150467ac65e05c42b4a1ce"
server_key_version: "encryptionkey@0"
I20260811 09:06:11.697839 29758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:11.727062 29758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:11.727989 29758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:11.728305 29758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:11.748379 29758 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44157
I20260811 09:06:11.748400 29835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44157 every 8 connection(s)
I20260811 09:06:11.749836 29758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:11.753738 29836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:11.758426 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29758
I20260811 09:06:11.758621 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:11.759087 9377 external_mini_cluster.cc:1428] Setting key 9daeee024760ac8ad5e92f11fde11b5a
I20260811 09:06:11.766937 29836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: Bootstrap starting.
I20260811 09:06:11.770866 29836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:11.772251 29836 log.cc:824] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:11.775631 29839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:11.761153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.9.40.126:44157)
0811 09:06:11.762310 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:11.762325 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:11.762382 (+ 57us) server_negotiation.cc:407] Connection header received
0811 09:06:11.764987 (+ 2605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:11.765017 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:11.765616 (+ 599us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:11.766058 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:11.766630 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:11.768660 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:11.770215 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:11.771405 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:11.771581 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:11.772824 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:11.772838 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:11.772861 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:11.773533 (+ 672us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:11.773650 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:11.773677 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:11.773802 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:11.773998 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 09:06:11.774363 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":360,"threads_started":1}
I20260811 09:06:11.775768 29836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: No bootstrap required, opened a new log
I20260811 09:06:11.784224 29836 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } }
I20260811 09:06:11.784593 29836 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:11.784678 29836 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2b373cfd55f44719471a2c4a9299064, State: Initialized, Role: FOLLOWER
I20260811 09:06:11.785312 29836 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } }
I20260811 09:06:11.785531 29836 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:11.785640 29836 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:11.785763 29836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:11.788067 29836 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } }
I20260811 09:06:11.788617 29836 leader_election.cc:302] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2b373cfd55f44719471a2c4a9299064; no voters:
I20260811 09:06:11.789247 29836 leader_election.cc:288] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:11.789405 29841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:11.790140 29841 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [term 1 LEADER]: Becoming Leader. State: Replica: b2b373cfd55f44719471a2c4a9299064, State: Running, Role: LEADER
I20260811 09:06:11.790732 29841 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } }
I20260811 09:06:11.791363 29836 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:11.794627 29842 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2b373cfd55f44719471a2c4a9299064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } } }
I20260811 09:06:11.794656 29843 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2b373cfd55f44719471a2c4a9299064. Latest consensus state: current_term: 1 leader_uuid: "b2b373cfd55f44719471a2c4a9299064" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2b373cfd55f44719471a2c4a9299064" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44157 } } }
I20260811 09:06:11.795200 29842 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:11.795236 29843 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:11.797636 29850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:11.803072 29850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:11.811913 29850 catalog_manager.cc:1347] Generated new cluster ID: 95816978c271401cb63215a3b5f44345
I20260811 09:06:11.812036 29850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:11.828267 29850 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:11.829703 29850 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:11.840652 29850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: Generated new TSK 0
I20260811 09:06:11.841642 29850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:11.852543 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--builtin_ntp_servers=127.9.40.84:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:12.237896 29861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:12.238230 29861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:12.238320 29861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:12.245950 29861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:12.246075 29861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:12.246186 29861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:12.255820 29861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:12.257308 29861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:12.259234 29861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:12.273389 29870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:12.273351 29867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:12.273432 29868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:12.274555 29861 server_base.cc:1034] running on GCE node
I20260811 09:06:12.275336 29861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:12.276360 29861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:12.277590 29861 hybrid_clock.cc:648] HybridClock initialized: now 1786439172277505 us; error 91 us; skew 500 ppm
I20260811 09:06:12.277959 29861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:12.280450 29861 webserver.cc:460] Webserver started at http://127.9.40.65:41395/ using document root <none> and password file <none>
I20260811 09:06:12.281301 29861 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:12.281428 29861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:12.281796 29861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:12.285025 29861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "b6321211ab0949f195ff8aaf2f3051f3"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "d1ba645b69c1e71453296e1b784fb8aa"
server_key_iv: "15e516b89d9955a9c927e4bf0a831e33"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.285889 29861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "b6321211ab0949f195ff8aaf2f3051f3"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "d1ba645b69c1e71453296e1b784fb8aa"
server_key_iv: "15e516b89d9955a9c927e4bf0a831e33"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.291445 29861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:06:12.295271 29876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.296952 29861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:06:12.297123 29861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "b6321211ab0949f195ff8aaf2f3051f3"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "d1ba645b69c1e71453296e1b784fb8aa"
server_key_iv: "15e516b89d9955a9c927e4bf0a831e33"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.297379 29861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:12.312101 29861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:12.313136 29861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:12.313464 29861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:12.314586 29861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:12.316262 29861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:12.316355 29861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.316517 29861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:12.316622 29861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.353231 29861 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45629
I20260811 09:06:12.353289 30005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45629 every 8 connection(s)
I20260811 09:06:12.354722 29861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:12.363262 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 29861
I20260811 09:06:12.363456 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:12.364019 9377 external_mini_cluster.cc:1428] Setting key fb904e7143ebcd3e7903443152659280
I20260811 09:06:12.367205 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--builtin_ntp_servers=127.9.40.84:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:12.369117 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:12.357561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56573 (local address 127.9.40.126:44157)
0811 09:06:12.358070 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:12.358074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:12.359282 (+ 1208us) server_negotiation.cc:407] Connection header received
0811 09:06:12.361190 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:12.361195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:12.361269 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:12.361374 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:12.362571 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.363643 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.365734 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.366248 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.366379 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:12.367987 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:12.367997 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:12.368005 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:12.368032 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:12.368102 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:12.368134 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:12.368237 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:12.368483 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 09:06:12.368606 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":252,"threads_started":1}
I20260811 09:06:12.370082 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:12.358177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44157 (local address 127.9.40.65:56573)
0811 09:06:12.359111 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:12.359148 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:12.360914 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:12.361668 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:12.361681 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:12.361777 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:12.362400 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:12.362417 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.363867 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.363872 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:12.365569 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:12.365594 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.366489 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.366494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:12.366758 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:12.367617 (+ 859us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:12.367620 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:12.367667 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:12.367761 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:12.368252 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:12.368265 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:12.368272 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:06:12.368434 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 09:06:12.368876 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":297,"threads_started":1}
I20260811 09:06:12.371925 30006 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44157
I20260811 09:06:12.372255 30006 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:12.373180 30006 heartbeater.cc:511] Master 127.9.40.126:44157 requested a full tablet report, sending...
I20260811 09:06:12.375463 29797 ts_manager.cc:194] Registered new tserver with Master: b6321211ab0949f195ff8aaf2f3051f3 (127.9.40.65:45629)
I20260811 09:06:12.377107 29797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:56573
W20260811 09:06:12.731098 30011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:12.731431 30011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:12.731602 30011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:12.739166 30011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:12.739286 30011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:12.739411 30011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:12.748922 30011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:12.750209 30011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:12.752127 30011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:12.767339 30016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:12.767315 30017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:12.767282 30019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:12.768103 30011 server_base.cc:1034] running on GCE node
I20260811 09:06:12.768874 30011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:12.769886 30011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:12.771220 30011 hybrid_clock.cc:648] HybridClock initialized: now 1786439172771129 us; error 54 us; skew 500 ppm
I20260811 09:06:12.771595 30011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:12.774308 30011 webserver.cc:460] Webserver started at http://127.9.40.66:44565/ using document root <none> and password file <none>
I20260811 09:06:12.775063 30011 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:12.775223 30011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:12.775588 30011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:12.778834 30011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "ff8efe05f1914ccbb9e862220ef1335e"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "9ad4e08a56822d051b6d57842e771bee"
server_key_iv: "6feb44ae029e846f9e3d12f008d0a700"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.779707 30011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "ff8efe05f1914ccbb9e862220ef1335e"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "9ad4e08a56822d051b6d57842e771bee"
server_key_iv: "6feb44ae029e846f9e3d12f008d0a700"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.785295 30011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:06:12.788936 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.790289 30011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:12.790479 30011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "ff8efe05f1914ccbb9e862220ef1335e"
format_stamp: "Formatted at 2026-08-11 09:06:12 on dist-test-slave-cbsf"
server_key: "9ad4e08a56822d051b6d57842e771bee"
server_key_iv: "6feb44ae029e846f9e3d12f008d0a700"
server_key_version: "encryptionkey@0"
I20260811 09:06:12.790715 30011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:12.815253 30011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:12.816193 30011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:12.816502 30011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:12.817673 30011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:12.819285 30011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:12.819413 30011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.819556 30011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:12.819653 30011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:12.854888 30011 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:43287
I20260811 09:06:12.854943 30157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:43287 every 8 connection(s)
I20260811 09:06:12.856280 30011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:12.864535 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30011
I20260811 09:06:12.864774 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:12.865262 9377 external_mini_cluster.cc:1428] Setting key b0fecaa07ca8072f31477dae045d31c4
I20260811 09:06:12.868592 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--builtin_ntp_servers=127.9.40.84:40257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:12.870968 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:12.859246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:40181 (local address 127.9.40.126:44157)
0811 09:06:12.859443 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:12.859448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:12.860985 (+ 1537us) server_negotiation.cc:407] Connection header received
0811 09:06:12.862924 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:12.862931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:12.863037 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:12.863152 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:12.864241 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.865223 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.867392 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.868029 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.868174 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:12.869951 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:12.869960 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:12.869969 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:12.869996 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:12.870042 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:12.870061 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:12.870159 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:12.870413 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 09:06:12.870536 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:06:12.872008 30160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:12.859783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44157 (local address 127.9.40.66:40181)
0811 09:06:12.860800 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:12.860850 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:12.862635 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:12.863387 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:12.863395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:12.863496 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:12.864066 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:12.864083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.865392 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.865397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:12.867250 (+ 1853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:12.867265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:12.868306 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:12.868315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:12.868651 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:12.869530 (+ 879us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:12.869562 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:12.869607 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:12.869721 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:12.870191 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:12.870200 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:12.870207 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:06:12.870359 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 09:06:12.870816 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":306,"threads_started":1}
I20260811 09:06:12.873638 30158 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44157
I20260811 09:06:12.873994 30158 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:12.874956 30158 heartbeater.cc:511] Master 127.9.40.126:44157 requested a full tablet report, sending...
I20260811 09:06:12.876628 29797 ts_manager.cc:194] Registered new tserver with Master: ff8efe05f1914ccbb9e862220ef1335e (127.9.40.66:43287)
I20260811 09:06:12.877516 29797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:40181
W20260811 09:06:13.229655 30161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:13.229988 30161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:13.230080 30161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:13.237773 30161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:13.237900 30161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:13.238016 30161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:13.247696 30161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44157
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:13.249114 30161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:13.250944 30161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:13.265141 30168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:13.265100 30169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:13.265108 30171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:13.266413 30161 server_base.cc:1034] running on GCE node
I20260811 09:06:13.267098 30161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:13.268160 30161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:13.269388 30161 hybrid_clock.cc:648] HybridClock initialized: now 1786439173269340 us; error 31 us; skew 500 ppm
I20260811 09:06:13.269752 30161 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:13.272303 30161 webserver.cc:460] Webserver started at http://127.9.40.67:40099/ using document root <none> and password file <none>
I20260811 09:06:13.273108 30161 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:13.273206 30161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:13.273619 30161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:13.276789 30161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "39f426a0cd494c4880e6d51154f08994"
format_stamp: "Formatted at 2026-08-11 09:06:13 on dist-test-slave-cbsf"
server_key: "318ae68769fdff86638405a06127512a"
server_key_iv: "9996f22c0dc6cf2cd6ccebc690aa986c"
server_key_version: "encryptionkey@0"
I20260811 09:06:13.277786 30161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "39f426a0cd494c4880e6d51154f08994"
format_stamp: "Formatted at 2026-08-11 09:06:13 on dist-test-slave-cbsf"
server_key: "318ae68769fdff86638405a06127512a"
server_key_iv: "9996f22c0dc6cf2cd6ccebc690aa986c"
server_key_version: "encryptionkey@0"
I20260811 09:06:13.283353 30161 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:06:13.287334 30179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:13.288950 30161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:06:13.289145 30161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "39f426a0cd494c4880e6d51154f08994"
format_stamp: "Formatted at 2026-08-11 09:06:13 on dist-test-slave-cbsf"
server_key: "318ae68769fdff86638405a06127512a"
server_key_iv: "9996f22c0dc6cf2cd6ccebc690aa986c"
server_key_version: "encryptionkey@0"
I20260811 09:06:13.289358 30161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:13.312099 30161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:13.313131 30161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:13.313458 30161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:13.314524 30161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:13.316224 30161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:13.316344 30161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:13.316491 30161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:13.316589 30161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:13.352455 30161 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:45343
I20260811 09:06:13.352488 30306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:45343 every 8 connection(s)
I20260811 09:06:13.354295 30161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:13.354708 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30161
I20260811 09:06:13.354907 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:13.355543 9377 external_mini_cluster.cc:1428] Setting key 1ba0ccad43d7d5ac49ae2f8a4b0d7b00
I20260811 09:06:13.369437 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.357429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37409 (local address 127.9.40.126:44157)
0811 09:06:13.357686 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.357696 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.360264 (+ 2568us) server_negotiation.cc:407] Connection header received
0811 09:06:13.362598 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.362615 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.362713 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.362833 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:13.363930 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.364770 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.366378 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.366814 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.366957 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.368470 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:13.368475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:13.368478 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:13.368500 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:13.368533 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:13.368546 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:13.368655 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.368965 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.369078 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:06:13.370760 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.358200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44157 (local address 127.9.40.67:37409)
0811 09:06:13.360087 (+ 1887us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.360150 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.362316 (+ 2166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.363074 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.363083 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.363185 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:13.363754 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.363771 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.364936 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.364940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.366207 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.366219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.367060 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.367067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.367349 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.368113 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:13.368116 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:13.368159 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:13.368250 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:13.368655 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:13.368669 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:13.368680 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.368938 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.369420 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1316,"thread_start_us":1086,"threads_started":1}
I20260811 09:06:13.372471 30307 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44157
I20260811 09:06:13.372855 30307 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:13.373744 30307 heartbeater.cc:511] Master 127.9.40.126:44157 requested a full tablet report, sending...
I20260811 09:06:13.375378 29797 ts_manager.cc:194] Registered new tserver with Master: 39f426a0cd494c4880e6d51154f08994 (127.9.40.67:45343)
I20260811 09:06:13.376207 29797 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:37409
I20260811 09:06:13.380378 30006 heartbeater.cc:503] Master 127.9.40.126:44157 was elected leader, sending a full tablet report...
I20260811 09:06:13.384923 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:13.395573 30008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.389315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.9.40.126:44157)
0811 09:06:13.389523 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.389528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.389968 (+ 440us) server_negotiation.cc:407] Connection header received
0811 09:06:13.390125 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.390129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.390194 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.390279 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:13.390747 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.391540 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.393264 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.393768 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.393867 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.394975 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:13.394980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:13.394984 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:13.395002 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:13.395036 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:13.395048 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:13.395130 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.395258 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.395338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:06:13.399611 29797 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:13.407213 29797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:13.430368 30322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.419742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43287 (local address 127.0.0.1:41122)
0811 09:06:13.420968 (+ 1226us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.421028 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.421208 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.422334 (+ 1126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.422341 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.422378 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:13.422864 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.422876 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.424686 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.424723 (+ 37us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.427014 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.427035 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.429318 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.429324 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.429554 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.429619 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.429750 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.429967 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":108,"thread_start_us":741,"threads_started":1}
I20260811 09:06:13.430390 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.420167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.9.40.66:43287)
0811 09:06:13.421155 (+ 988us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.421163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.421194 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:06:13.421548 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.421556 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.421830 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.422017 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:13.423109 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.424509 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.427257 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.428757 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.428894 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.429182 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.429824 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.430016 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":810,"thread_start_us":693,"threads_started":1}
I20260811 09:06:13.430619 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.419851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.9.40.67:45343)
0811 09:06:13.420539 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.420550 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.421777 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 09:06:13.422093 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.422104 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.422327 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.422492 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:13.423582 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.425203 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.427806 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.429298 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.429424 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.429699 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.430033 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.430201 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":324,"threads_started":1}
I20260811 09:06:13.431231 30324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.419742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:45343 (local address 127.0.0.1:45212)
0811 09:06:13.421662 (+ 1920us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.421680 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.421884 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.422846 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.422850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.422874 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:13.423293 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.423305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.425383 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.425393 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.427564 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.427579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.429544 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.429550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.429730 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.429764 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.429835 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.430714 (+ 879us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1780,"mutex_wait_us":230,"thread_start_us":1352,"threads_started":1}
I20260811 09:06:13.431752 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.421728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53362)
0811 09:06:13.422687 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.422706 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.422859 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.424129 (+ 1270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.424135 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.424161 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:13.424531 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.424542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.426846 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.426853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.428988 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.429003 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.430947 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.430955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.431176 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.431230 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.431302 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.431466 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":827,"thread_start_us":843,"threads_started":1}
I20260811 09:06:13.432644 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.422346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53362 (local address 127.9.40.65:45629)
0811 09:06:13.423535 (+ 1189us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.423543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.423572 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:06:13.423670 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.423677 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.423908 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.424135 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:13.425053 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.426686 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.429222 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.430715 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.431661 (+ 946us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.431965 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.432047 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.432241 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":847,"threads_started":1}
I20260811 09:06:13.435122 30228 tablet_service.cc:1467] Processing CreateTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd (DEFAULT_TABLE table=test-table [id=616cfebf27f1449d9041dccfa89a7072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:13.435310 30086 tablet_service.cc:1467] Processing CreateTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd (DEFAULT_TABLE table=test-table [id=616cfebf27f1449d9041dccfa89a7072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:13.436517 30228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72b4d1ebd4241f89fcf4c08615e53bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:13.436762 30086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72b4d1ebd4241f89fcf4c08615e53bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:13.436399 29933 tablet_service.cc:1467] Processing CreateTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd (DEFAULT_TABLE table=test-table [id=616cfebf27f1449d9041dccfa89a7072]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:13.437834 29933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e72b4d1ebd4241f89fcf4c08615e53bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:13.449910 30328 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Bootstrap starting.
I20260811 09:06:13.453224 30329 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Bootstrap starting.
I20260811 09:06:13.455144 30330 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: Bootstrap starting.
I20260811 09:06:13.455189 30328 tablet_bootstrap.cc:654] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:13.456560 30328 log.cc:824] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:13.457223 30329 tablet_bootstrap.cc:654] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:13.458477 30329 log.cc:824] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:13.458657 30330 tablet_bootstrap.cc:654] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:13.460006 30330 log.cc:824] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:13.460176 30328 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: No bootstrap required, opened a new log
I20260811 09:06:13.460493 30328 ts_tablet_manager.cc:1397] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 09:06:13.462121 30329 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: No bootstrap required, opened a new log
I20260811 09:06:13.462412 30329 ts_tablet_manager.cc:1397] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:06:13.463213 30330 tablet_bootstrap.cc:492] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: No bootstrap required, opened a new log
I20260811 09:06:13.463490 30330 ts_tablet_manager.cc:1397] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260811 09:06:13.471257 30330 raft_consensus.cc:348] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.471721 30330 raft_consensus.cc:374] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:13.471851 30330 raft_consensus.cc:729] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6321211ab0949f195ff8aaf2f3051f3, State: Initialized, Role: FOLLOWER
I20260811 09:06:13.472594 30330 consensus_queue.cc:260] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.472654 30328 raft_consensus.cc:348] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.473181 30328 raft_consensus.cc:374] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:13.473312 30328 raft_consensus.cc:729] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f426a0cd494c4880e6d51154f08994, State: Initialized, Role: FOLLOWER
I20260811 09:06:13.473518 30329 raft_consensus.cc:348] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.473982 30329 raft_consensus.cc:374] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:13.474116 30329 raft_consensus.cc:729] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8efe05f1914ccbb9e862220ef1335e, State: Initialized, Role: FOLLOWER
I20260811 09:06:13.474275 30328 consensus_queue.cc:260] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.474817 30329 consensus_queue.cc:260] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.476142 30330 ts_tablet_manager.cc:1428] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
I20260811 09:06:13.477288 30158 heartbeater.cc:503] Master 127.9.40.126:44157 was elected leader, sending a full tablet report...
I20260811 09:06:13.477811 30307 heartbeater.cc:503] Master 127.9.40.126:44157 was elected leader, sending a full tablet report...
I20260811 09:06:13.478425 30328 ts_tablet_manager.cc:1428] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Time spent starting tablet: real 0.018s user 0.008s sys 0.007s
I20260811 09:06:13.479660 30329 ts_tablet_manager.cc:1428] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Time spent starting tablet: real 0.017s user 0.010s sys 0.004s
I20260811 09:06:13.493594 30337 raft_consensus.cc:482] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:13.494021 30337 raft_consensus.cc:504] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.496125 30337 leader_election.cc:288] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff8efe05f1914ccbb9e862220ef1335e (127.9.40.66:43287), b6321211ab0949f195ff8aaf2f3051f3 (127.9.40.65:45629)
I20260811 09:06:13.504097 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.496623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34607 (local address 127.9.40.66:43287)
0811 09:06:13.496824 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.496830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.497520 (+ 690us) server_negotiation.cc:407] Connection header received
0811 09:06:13.497668 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.497672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.497762 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.497894 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:13.498572 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.499375 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.501853 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.503079 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.503177 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.503245 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.503668 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.503803 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 09:06:13.504223 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.496535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43287 (local address 127.9.40.67:34607)
0811 09:06:13.497384 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.497401 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.497549 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.497937 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.497977 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.498008 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:13.498400 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.498412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.499561 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.499576 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.501630 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.501648 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.503248 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.503255 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.503441 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.503481 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.503596 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.503758 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":22,"thread_start_us":512,"threads_started":1}
I20260811 09:06:13.504621 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.496461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.9.40.67:40079)
0811 09:06:13.496863 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.496894 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:13.497085 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:13.497606 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:13.497612 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:13.497643 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:13.497852 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.497867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.499158 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.499164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.501090 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:13.501126 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.502855 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.502862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:13.503694 (+ 832us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.503731 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:06:13.503820 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 09:06:13.503948 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":72}
I20260811 09:06:13.505021 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.496584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:40079 (local address 127.9.40.65:45629)
0811 09:06:13.496817 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.496825 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.497009 (+ 184us) server_negotiation.cc:407] Connection header received
0811 09:06:13.497287 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.497294 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.497397 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.497511 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:13.498047 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.499022 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.501305 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.502680 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.502843 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.502976 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.504515 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.504676 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:06:13.505539 30106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72b4d1ebd4241f89fcf4c08615e53bd" candidate_uuid: "39f426a0cd494c4880e6d51154f08994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff8efe05f1914ccbb9e862220ef1335e" is_pre_election: true
I20260811 09:06:13.505929 29954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72b4d1ebd4241f89fcf4c08615e53bd" candidate_uuid: "39f426a0cd494c4880e6d51154f08994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6321211ab0949f195ff8aaf2f3051f3" is_pre_election: true
I20260811 09:06:13.506129 30106 raft_consensus.cc:2454] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f426a0cd494c4880e6d51154f08994 in term 0.
I20260811 09:06:13.506470 29954 raft_consensus.cc:2454] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 39f426a0cd494c4880e6d51154f08994 in term 0.
I20260811 09:06:13.507068 30180 leader_election.cc:302] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f426a0cd494c4880e6d51154f08994, ff8efe05f1914ccbb9e862220ef1335e; no voters:
I20260811 09:06:13.507572 30337 raft_consensus.cc:2781] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:13.507743 30337 raft_consensus.cc:482] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:13.507872 30337 raft_consensus.cc:3037] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:13.510360 30337 raft_consensus.cc:504] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.511009 30337 leader_election.cc:288] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [CANDIDATE]: Term 1 election: Requested vote from peers ff8efe05f1914ccbb9e862220ef1335e (127.9.40.66:43287), b6321211ab0949f195ff8aaf2f3051f3 (127.9.40.65:45629)
I20260811 09:06:13.511555 29954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72b4d1ebd4241f89fcf4c08615e53bd" candidate_uuid: "39f426a0cd494c4880e6d51154f08994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6321211ab0949f195ff8aaf2f3051f3"
I20260811 09:06:13.511556 30106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e72b4d1ebd4241f89fcf4c08615e53bd" candidate_uuid: "39f426a0cd494c4880e6d51154f08994" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff8efe05f1914ccbb9e862220ef1335e"
I20260811 09:06:13.511788 29954 raft_consensus.cc:3037] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:13.511812 30106 raft_consensus.cc:3037] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:13.514231 29954 raft_consensus.cc:2454] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f426a0cd494c4880e6d51154f08994 in term 1.
I20260811 09:06:13.514245 30106 raft_consensus.cc:2454] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 39f426a0cd494c4880e6d51154f08994 in term 1.
I20260811 09:06:13.514763 30181 leader_election.cc:302] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39f426a0cd494c4880e6d51154f08994, b6321211ab0949f195ff8aaf2f3051f3; no voters:
I20260811 09:06:13.515139 30337 raft_consensus.cc:2781] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:13.515671 30337 raft_consensus.cc:686] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 1 LEADER]: Becoming Leader. State: Replica: 39f426a0cd494c4880e6d51154f08994, State: Running, Role: LEADER
I20260811 09:06:13.516265 30337 consensus_queue.cc:237] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } }
I20260811 09:06:13.521507 29795 catalog_manager.cc:5495] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 reported cstate change: term changed from 0 to 1, leader changed from <none> to 39f426a0cd494c4880e6d51154f08994 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "39f426a0cd494c4880e6d51154f08994" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "39f426a0cd494c4880e6d51154f08994" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45343 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 } health_report { overall_health: UNKNOWN } } }
I20260811 09:06:13.561404 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.555652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45218 (local address 127.9.40.67:45343)
0811 09:06:13.555833 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.555838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.555855 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:06:13.555949 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.555953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.556033 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.556175 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:13.556512 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.557349 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.559049 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.559470 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.559563 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.560153 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:13.560750 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:13.560868 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.561002 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.561111 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:06:13.570081 29954 raft_consensus.cc:1261] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3 [term 1 FOLLOWER]: Refusing update from remote peer 39f426a0cd494c4880e6d51154f08994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:13.570081 30106 raft_consensus.cc:1261] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 1 FOLLOWER]: Refusing update from remote peer 39f426a0cd494c4880e6d51154f08994: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:13.571122 30340 consensus_queue.cc:1035] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff8efe05f1914ccbb9e862220ef1335e" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:13.571538 30337 consensus_queue.cc:1035] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6321211ab0949f195ff8aaf2f3051f3" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:13.601895 29795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:13.602245 29795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:13.606314 29795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b2b373cfd55f44719471a2c4a9299064: Sending DeleteTablet for 3 replicas of tablet e72b4d1ebd4241f89fcf4c08615e53bd
I20260811 09:06:13.607712 30086 tablet_service.cc:1514] Processing DeleteTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:13 UTC) from {username='slave'} at 127.0.0.1:41122
I20260811 09:06:13.607788 30228 tablet_service.cc:1514] Processing DeleteTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:13 UTC) from {username='slave'} at 127.0.0.1:45212
I20260811 09:06:13.607839 29933 tablet_service.cc:1514] Processing DeleteTablet for tablet e72b4d1ebd4241f89fcf4c08615e53bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:13 UTC) from {username='slave'} at 127.0.0.1:53362
I20260811 09:06:13.610602 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29861
I20260811 09:06:13.644472 30355 tablet_replica.cc:330] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: stopping tablet replica
I20260811 09:06:13.645479 30355 raft_consensus.cc:2229] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:13.646445 30355 raft_consensus.cc:2258] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:13.652992 30355 ts_tablet_manager.cc:1905] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:13.656195 30327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.612842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.9.40.65:45629)
0811 09:06:13.613105 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.613116 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.636646 (+ 23530us) server_negotiation.cc:407] Connection header received
0811 09:06:13.637500 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.637511 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.637699 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.637820 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:13.638997 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.640755 (+ 1758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.646792 (+ 6037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.647916 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.648076 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.653500 (+ 5424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:13.653520 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:13.653549 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:13.654674 (+ 1125us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:13.654773 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:13.654813 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:13.654939 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.655328 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.655546 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:06:13.658339 30000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:13.664922 30353 tablet_replica.cc:330] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: stopping tablet replica
I20260811 09:06:13.665745 30353 raft_consensus.cc:2229] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:13.666982 30353 raft_consensus.cc:2258] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:13.669735 30353 ts_tablet_manager.cc:1905] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:13.677343 30355 ts_tablet_manager.cc:1918] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:13.677567 30355 log.cc:1196] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/e72b4d1ebd4241f89fcf4c08615e53bd
I20260811 09:06:13.678210 30355 ts_tablet_manager.cc:1939] T e72b4d1ebd4241f89fcf4c08615e53bd P ff8efe05f1914ccbb9e862220ef1335e: Deleting consensus metadata
I20260811 09:06:13.680462 29777 catalog_manager.cc:4841] TS ff8efe05f1914ccbb9e862220ef1335e (127.9.40.66:43287): tablet e72b4d1ebd4241f89fcf4c08615e53bd (table test-table [id=616cfebf27f1449d9041dccfa89a7072]) successfully deleted
I20260811 09:06:13.683521 30353 ts_tablet_manager.cc:1918] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:13.683765 30353 log.cc:1196] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/e72b4d1ebd4241f89fcf4c08615e53bd
I20260811 09:06:13.684486 30353 ts_tablet_manager.cc:1939] T e72b4d1ebd4241f89fcf4c08615e53bd P 39f426a0cd494c4880e6d51154f08994: Deleting consensus metadata
I20260811 09:06:13.686480 29779 catalog_manager.cc:4841] TS 39f426a0cd494c4880e6d51154f08994 (127.9.40.67:45343): tablet e72b4d1ebd4241f89fcf4c08615e53bd (table test-table [id=616cfebf27f1449d9041dccfa89a7072]) successfully deleted
I20260811 09:06:13.786077 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29861
I20260811 09:06:13.786154 30354 tablet_replica.cc:330] T e72b4d1ebd4241f89fcf4c08615e53bd P b6321211ab0949f195ff8aaf2f3051f3: stopping tablet replica
I20260811 09:06:13.815629 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30011
W20260811 09:06:13.815761 29779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:45629 (error 108)
I20260811 09:06:13.819360 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.818907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53388)
0811 09:06:13.819106 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.819213 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:06:13.819795 29779 catalog_manager.cc:4584] TS b6321211ab0949f195ff8aaf2f3051f3 (127.9.40.65:45629): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e72b4d1ebd4241f89fcf4c08615e53bd: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
I20260811 09:06:13.825126 30325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.817025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41136 (local address 127.9.40.66:43287)
0811 09:06:13.817285 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.817294 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.817322 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:06:13.817411 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.817419 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.817561 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.817711 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:13.818312 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.819571 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.821343 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.822083 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.822253 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.823322 (+ 1069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:13.823335 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:13.823354 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:13.824212 (+ 858us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:13.824282 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:13.824312 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:13.824450 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.824556 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.824707 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:06:13.826022 30153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:13.873715 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.873112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53396)
0811 09:06:13.873379 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.873499 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 09:06:13.875025 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.874576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53404)
0811 09:06:13.874757 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.874861 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 09:06:13.918865 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30011
I20260811 09:06:13.935817 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.935324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53418)
0811 09:06:13.935547 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.935661 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 09:06:13.936959 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.936574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53434)
0811 09:06:13.936760 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:13.936844 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:06:13.950253 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30161
I20260811 09:06:13.959007 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:13.951377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.9.40.67:45343)
0811 09:06:13.951608 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:13.951614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:13.951637 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:06:13.951714 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:13.951720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:13.951831 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:13.951965 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:13.952339 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.953574 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.954976 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:13.955696 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:13.955839 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:13.957152 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:13.957172 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:13.957200 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:13.958085 (+ 885us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:13.958148 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:13.958174 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:13.958291 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:13.958464 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:06:13.958613 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:06:13.959947 30302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:14.017974 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:14.017386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53442)
0811 09:06:14.017664 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:14.017777 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 09:06:14.019266 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:14.018784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53450)
0811 09:06:14.019039 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:14.019152 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260811 09:06:14.083122 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30161
I20260811 09:06:14.108517 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 29758
I20260811 09:06:14.109778 29830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:14.223801 30326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:14.223147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45629 (local address 127.0.0.1:53454)
0811 09:06:14.223417 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:14.223531 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 09:06:14.223800 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 29758
2026-08-11T09:06:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3051 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:14 dist-test-slave-cbsf krb5kdc[30364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:14 dist-test-slave-cbsf krb5kdc[30364](info): set up 2 sockets
Aug 11 09:06:14 dist-test-slave-cbsf krb5kdc[30364](info): commencing operation
krb5kdc: starting...
W20260811 09:06:15.464494 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.006s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:15 dist-test-slave-cbsf krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:06:15.808614 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33263
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:16.171578 30380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:16.171909 30380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:16.172103 30380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:16.179937 30380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:16.180068 30380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:16.180154 30380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:16.180262 30380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:16.180351 30380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:16.191082 30380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:16.192575 30380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:16.193954 30380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:16.206804 30385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:16.206771 30386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:16.206719 30389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:16.208184 30380 server_base.cc:1034] running on GCE node
I20260811 09:06:16.209069 30380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:16.210493 30380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:16.211723 30380 hybrid_clock.cc:648] HybridClock initialized: now 1786439176211684 us; error 34 us; skew 500 ppm
I20260811 09:06:16.212153 30380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:16 dist-test-slave-cbsf krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:16.215663 30380 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:06:16.217334 30380 webserver.cc:460] Webserver started at http://127.9.40.126:46033/ using document root <none> and password file <none>
I20260811 09:06:16.218063 30380 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:16.218220 30380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:16.218549 30380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:16.221843 30380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "35517a069ade484788d732706fa67b54"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c087cc31abbe48a5e47adb552ecb9cc0"
server_key_iv: "f9986022b3cfabe3e1eeafc93a29b266"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.222692 30380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "35517a069ade484788d732706fa67b54"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c087cc31abbe48a5e47adb552ecb9cc0"
server_key_iv: "f9986022b3cfabe3e1eeafc93a29b266"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.228619 30380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:06:16.232583 30397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:16.234117 30380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:16.234329 30380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "35517a069ade484788d732706fa67b54"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c087cc31abbe48a5e47adb552ecb9cc0"
server_key_iv: "f9986022b3cfabe3e1eeafc93a29b266"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.234603 30380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:16.255810 30380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:16.260492 30380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:16.260865 30380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:16.281309 30380 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33263
I20260811 09:06:16.281404 30458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33263 every 8 connection(s)
I20260811 09:06:16.282809 30380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:16.285100 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30380
I20260811 09:06:16.285311 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:16.285990 9377 external_mini_cluster.cc:1428] Setting key eaade61b8194628fce50f17f04e1b6ea
I20260811 09:06:16.287642 30459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:06:16 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:16.300921 30459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: Bootstrap starting.
I20260811 09:06:16.304806 30459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:16.306021 30459 log.cc:824] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:16.309826 30459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: No bootstrap required, opened a new log
I20260811 09:06:16.310449 30462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:16.288112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49618 (local address 127.9.40.126:33263)
0811 09:06:16.289048 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:16.289060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:16.289100 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:06:16.290732 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:16.290754 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:16.291153 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:16.291574 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:16.293237 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:16.294616 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:16.296000 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:16.296681 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:16.296964 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:16.301781 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:16.301789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:16.301805 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:16.301844 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:16.305389 (+ 3545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:16.306155 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:16.306163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:16.306170 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:16.306389 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:16.306903 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:16.306907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:16.306910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:16.307658 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:16.307983 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:16.308439 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 09:06:16.308985 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":362,"threads_started":1}
I20260811 09:06:16.318619 30459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } }
I20260811 09:06:16.319051 30459 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:16.319129 30459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35517a069ade484788d732706fa67b54, State: Initialized, Role: FOLLOWER
I20260811 09:06:16.319676 30459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } }
I20260811 09:06:16.319904 30459 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:16.320019 30459 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:16.320166 30459 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:16.322748 30459 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } }
I20260811 09:06:16.323302 30459 leader_election.cc:302] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35517a069ade484788d732706fa67b54; no voters:
I20260811 09:06:16.324026 30459 leader_election.cc:288] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:16.324173 30464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:16.329120 30464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [term 1 LEADER]: Becoming Leader. State: Replica: 35517a069ade484788d732706fa67b54, State: Running, Role: LEADER
I20260811 09:06:16.330024 30464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } }
I20260811 09:06:16.331175 30459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:16.333609 30466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35517a069ade484788d732706fa67b54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } } }
I20260811 09:06:16.334174 30466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:16.334774 30465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35517a069ade484788d732706fa67b54. Latest consensus state: current_term: 1 leader_uuid: "35517a069ade484788d732706fa67b54" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35517a069ade484788d732706fa67b54" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33263 } } }
I20260811 09:06:16.335289 30465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:16.336994 30472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:16.344331 30472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:16.354964 30472 catalog_manager.cc:1347] Generated new cluster ID: 69c3e5384e044575bdf0e6cab1c2f46f
I20260811 09:06:16.355106 30472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:16.368616 30472 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:16.370826 30472 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:16.392432 30472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: Generated new TSK 0
I20260811 09:06:16.393797 30472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:06:16.476622 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:16.870905 30490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:16.871219 30490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:16.871322 30490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:16.878983 30490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:16.879127 30490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:16.879245 30490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:16.889916 30490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:16.891327 30490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:16.892722 30490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:16.906708 30495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:16.906625 30496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:16.906675 30498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:16.907694 30490 server_base.cc:1034] running on GCE node
I20260811 09:06:16.908352 30490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:16.909409 30490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:16.910619 30490 hybrid_clock.cc:648] HybridClock initialized: now 1786439176910578 us; error 34 us; skew 500 ppm
I20260811 09:06:16.910960 30490 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:16 dist-test-slave-cbsf krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:16.919644 30490 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:06:16.921468 30490 webserver.cc:460] Webserver started at http://127.9.40.65:40751/ using document root <none> and password file <none>
I20260811 09:06:16.922241 30490 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:16.922384 30490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:16.922734 30490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:16.925944 30490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "5ca7ae58b04f46af83e686ea17d428d9"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c1555a745a9bc56231d305af28ae62d3"
server_key_iv: "c8bd1328347101f3c3fe762bb03eaa33"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.926796 30490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "5ca7ae58b04f46af83e686ea17d428d9"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c1555a745a9bc56231d305af28ae62d3"
server_key_iv: "c8bd1328347101f3c3fe762bb03eaa33"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.932421 30490 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 09:06:16.936049 30507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:16.937614 30490 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:16.937813 30490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "5ca7ae58b04f46af83e686ea17d428d9"
format_stamp: "Formatted at 2026-08-11 09:06:16 on dist-test-slave-cbsf"
server_key: "c1555a745a9bc56231d305af28ae62d3"
server_key_iv: "c8bd1328347101f3c3fe762bb03eaa33"
server_key_version: "encryptionkey@0"
I20260811 09:06:16.938098 30490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:16.958146 30490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:16.962689 30490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:16.963038 30490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:16.964146 30490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:16.965714 30490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:16.965826 30490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:16.965996 30490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:16.966075 30490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:17.004194 30490 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37023
I20260811 09:06:17.004236 30640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37023 every 8 connection(s)
I20260811 09:06:17.005600 30490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:17.016966 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30490
I20260811 09:06:17.017217 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:17.017673 9377 external_mini_cluster.cc:1428] Setting key eb7f705e70b1ef481bf92f85028448f9
Aug 11 09:06:17 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439176, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:17.029515 30644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:17.008329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42181 (local address 127.9.40.126:33263)
0811 09:06:17.008856 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:17.008861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:17.009771 (+ 910us) server_negotiation.cc:407] Connection header received
0811 09:06:17.012545 (+ 2774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:17.012552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:17.012641 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:17.012771 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:17.015020 (+ 2249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.016133 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.018604 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.019119 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.019272 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:17.023123 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:17.023138 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:17.023141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:17.023178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:17.026087 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:17.026952 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:17.026959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:17.026963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:17.027112 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:17.027577 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:17.027582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:17.027585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:17.027889 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:17.028066 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:17.028856 (+ 790us) server_negotiation.cc:299] Negotiation successful
0811 09:06:17.029098 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":282,"threads_started":1}
I20260811 09:06:17.030555 30645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:17.008746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33263 (local address 127.9.40.65:42181)
0811 09:06:17.009557 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:17.009611 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:17.012099 (+ 2488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:17.013209 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:17.013221 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:17.013973 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:17.014771 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:17.014795 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.016340 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.016346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:17.018407 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:17.018425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.019383 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.019393 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:17.019674 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:17.020435 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:17.020468 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:17.022864 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:17.026279 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:17.026291 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:17.026314 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:17.026799 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:17.027235 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:17.027240 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:17.027244 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:17.027447 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:17.028036 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:17.028044 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:17.028425 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 09:06:17.028740 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 09:06:17.029356 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":270,"threads_started":1}
I20260811 09:06:17.032241 30642 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33263
I20260811 09:06:17.032604 30642 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:17.033424 30642 heartbeater.cc:511] Master 127.9.40.126:33263 requested a full tablet report, sending...
I20260811 09:06:17.035749 30419 ts_manager.cc:194] Registered new tserver with Master: 5ca7ae58b04f46af83e686ea17d428d9 (127.9.40.65:37023)
I20260811 09:06:17.037467 30419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:42181
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:06:17.092790 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:17.454613 30649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:17.454962 30649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:17.455117 30649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:17.462776 30649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:17.462917 30649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:17.463129 30649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:17.473483 30649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:17.474963 30649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:17.476334 30649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:17.490134 30656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:17.490234 30657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:17.490273 30659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:17.491408 30649 server_base.cc:1034] running on GCE node
I20260811 09:06:17.492097 30649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:17.493196 30649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:17.494410 30649 hybrid_clock.cc:648] HybridClock initialized: now 1786439177494338 us; error 58 us; skew 500 ppm
I20260811 09:06:17.494776 30649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:17 dist-test-slave-cbsf krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439177, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:17.498344 30649 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:06:17.499890 30649 webserver.cc:460] Webserver started at http://127.9.40.66:46437/ using document root <none> and password file <none>
I20260811 09:06:17.500619 30649 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:17.500793 30649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:17.501127 30649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:17.504179 30649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "5cdeca0a713c45a78b13cee2a6575c5b"
format_stamp: "Formatted at 2026-08-11 09:06:17 on dist-test-slave-cbsf"
server_key: "2ed18d43aa73d17debec76a1e4a19878"
server_key_iv: "36439e336282701ed3ecbc1ed6d13394"
server_key_version: "encryptionkey@0"
I20260811 09:06:17.505040 30649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "5cdeca0a713c45a78b13cee2a6575c5b"
format_stamp: "Formatted at 2026-08-11 09:06:17 on dist-test-slave-cbsf"
server_key: "2ed18d43aa73d17debec76a1e4a19878"
server_key_iv: "36439e336282701ed3ecbc1ed6d13394"
server_key_version: "encryptionkey@0"
I20260811 09:06:17.510499 30649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:06:17.514133 30670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:17.515539 30649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:17.515748 30649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "5cdeca0a713c45a78b13cee2a6575c5b"
format_stamp: "Formatted at 2026-08-11 09:06:17 on dist-test-slave-cbsf"
server_key: "2ed18d43aa73d17debec76a1e4a19878"
server_key_iv: "36439e336282701ed3ecbc1ed6d13394"
server_key_version: "encryptionkey@0"
I20260811 09:06:17.515954 30649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:17.531922 30649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:17.536834 30649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:17.537163 30649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:17.538383 30649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:17.539875 30649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:17.539960 30649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:17.540048 30649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:17.540100 30649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:17.576629 30649 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35793
I20260811 09:06:17.576807 30799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35793 every 8 connection(s)
I20260811 09:06:17.578040 30649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:17.587990 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30649
I20260811 09:06:17.588213 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:17.588788 9377 external_mini_cluster.cc:1428] Setting key 04fba7698059fb57c1c65c8bce8bb252
Aug 11 09:06:17 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439177, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:17.599717 30802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:17.580836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56185 (local address 127.9.40.126:33263)
0811 09:06:17.581281 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:17.581286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:17.582234 (+ 948us) server_negotiation.cc:407] Connection header received
0811 09:06:17.584199 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:17.584204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:17.584296 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:17.584429 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:17.586052 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.586784 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.588451 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.589046 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.589189 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:17.593815 (+ 4626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:17.593821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:17.593826 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:17.593870 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:17.596457 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:17.597270 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:17.597275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:17.597279 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:17.597431 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:17.597883 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:17.597888 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:17.597891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:17.598180 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:17.598339 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:17.599066 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 09:06:17.599318 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":205,"threads_started":1}
I20260811 09:06:17.601886 30804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:17.581337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33263 (local address 127.9.40.66:56185)
0811 09:06:17.582067 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:17.582109 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:17.583915 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:17.584730 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:17.584743 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:17.585288 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:17.585876 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:17.585894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.586960 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.586964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:17.588309 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:17.588331 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:17.589302 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:17.589310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:17.589553 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:17.590302 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:17.590330 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:17.593574 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:17.596613 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:17.596623 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:17.596641 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:17.597128 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:17.597546 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:17.597551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:17.597555 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:17.597763 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:17.598313 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:17.598321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:17.598702 (+ 381us) client_negotiation.cc:769] Sending connection context
0811 09:06:17.598976 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 09:06:17.599537 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":292,"threads_started":1}
I20260811 09:06:17.603451 30800 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33263
I20260811 09:06:17.603797 30800 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:17.604749 30800 heartbeater.cc:511] Master 127.9.40.126:33263 requested a full tablet report, sending...
I20260811 09:06:17.606379 30419 ts_manager.cc:194] Registered new tserver with Master: 5cdeca0a713c45a78b13cee2a6575c5b (127.9.40.66:35793)
I20260811 09:06:17.607275 30419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:56185
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:06:17.666576 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:18.025368 30808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:18.025733 30808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:18.025905 30808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:18.033653 30808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:18.033818 30808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:18.034008 30808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:18.041571 30642 heartbeater.cc:503] Master 127.9.40.126:33263 was elected leader, sending a full tablet report...
I20260811 09:06:18.045383 30808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:18.046794 30808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:18.048167 30808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:18.062613 30815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:18.062690 30816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:18.062714 30819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:18.064203 30808 server_base.cc:1034] running on GCE node
I20260811 09:06:18.065043 30808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:18.066133 30808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:18.067402 30808 hybrid_clock.cc:648] HybridClock initialized: now 1786439178067325 us; error 71 us; skew 500 ppm
I20260811 09:06:18.067785 30808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:18 dist-test-slave-cbsf krb5kdc[30364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:18.071247 30808 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:06:18.072875 30808 webserver.cc:460] Webserver started at http://127.9.40.67:38881/ using document root <none> and password file <none>
I20260811 09:06:18.073603 30808 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:18.073709 30808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:18.074003 30808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:18.077075 30808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "236befc814aa4f859f2ffb3a562e3841"
format_stamp: "Formatted at 2026-08-11 09:06:18 on dist-test-slave-cbsf"
server_key: "bfa29148d0d60789424c3988174a911a"
server_key_iv: "9b365c7bb03a9e5c518be34f92d9e8c9"
server_key_version: "encryptionkey@0"
I20260811 09:06:18.077869 30808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "236befc814aa4f859f2ffb3a562e3841"
format_stamp: "Formatted at 2026-08-11 09:06:18 on dist-test-slave-cbsf"
server_key: "bfa29148d0d60789424c3988174a911a"
server_key_iv: "9b365c7bb03a9e5c518be34f92d9e8c9"
server_key_version: "encryptionkey@0"
I20260811 09:06:18.083410 30808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:06:18.087136 30829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:18.088619 30808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:06:18.088842 30808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "236befc814aa4f859f2ffb3a562e3841"
format_stamp: "Formatted at 2026-08-11 09:06:18 on dist-test-slave-cbsf"
server_key: "bfa29148d0d60789424c3988174a911a"
server_key_iv: "9b365c7bb03a9e5c518be34f92d9e8c9"
server_key_version: "encryptionkey@0"
I20260811 09:06:18.089030 30808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:18.155972 30808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:18.160543 30808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:18.160923 30808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:18.161921 30808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:18.163259 30808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:18.163336 30808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:18.163421 30808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:18.163466 30808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:18.198645 30808 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:45293
I20260811 09:06:18.198702 30963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:45293 every 8 connection(s)
I20260811 09:06:18.199977 30808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:18.202765 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 30808
I20260811 09:06:18.202979 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:18.203557 9377 external_mini_cluster.cc:1428] Setting key 9588bb62fafc2da3686613a23d60bb30
Aug 11 09:06:18 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439178, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:18.221995 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.202733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:47843 (local address 127.9.40.126:33263)
0811 09:06:18.203313 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.203321 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.204311 (+ 990us) server_negotiation.cc:407] Connection header received
0811 09:06:18.206940 (+ 2629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.206945 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.207036 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.207165 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:18.208734 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.209569 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.211124 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.211522 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.211657 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.215436 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:18.215441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:18.215444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:18.215477 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:18.218778 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.219785 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.219789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.219792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.219924 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.220349 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.220353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.220355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.220638 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:18.220851 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.221503 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.221706 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":290,"threads_started":1}
I20260811 09:06:18.223074 30968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.203296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33263 (local address 127.9.40.67:47843)
0811 09:06:18.204134 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.204182 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.206630 (+ 2448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.207420 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.207429 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.207943 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:18.208534 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.208556 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.209731 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.209737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.210953 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.210965 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.211747 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.211755 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.211997 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.212804 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:18.212828 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:18.215214 (+ 2386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:18.218924 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:18.218934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:18.218958 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:18.219659 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:18.220042 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:18.220045 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:18.220048 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:18.220250 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:18.220791 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:18.220797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:18.221165 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.221416 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.221851 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":334,"threads_started":1}
I20260811 09:06:18.224545 30964 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33263
I20260811 09:06:18.224900 30964 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:18.225685 30964 heartbeater.cc:511] Master 127.9.40.126:33263 requested a full tablet report, sending...
I20260811 09:06:18.227224 30419 ts_manager.cc:194] Registered new tserver with Master: 236befc814aa4f859f2ffb3a562e3841 (127.9.40.67:45293)
I20260811 09:06:18.228080 30419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:47843
I20260811 09:06:18.231767 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:18.251655 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.236552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.9.40.126:33263)
0811 09:06:18.236882 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.236893 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.237295 (+ 402us) server_negotiation.cc:407] Connection header received
0811 09:06:18.237482 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.237486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.237555 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.237642 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:18.239186 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.240066 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.241462 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.241878 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.242038 (+ 160us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.245033 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:18.245039 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:18.245042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:18.245081 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:18.247936 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.249003 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.249008 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.249011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.249163 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.249674 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.249678 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.249680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.250127 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:18.250502 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.250990 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.251242 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260811 09:06:18.255996 30419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:18.263195 30419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:18.309732 30980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.277052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35793 (local address 127.0.0.1:53578)
0811 09:06:18.298406 (+ 21354us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.298441 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.298589 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.299892 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.299902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.299946 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:18.300585 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.300603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.303615 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.303620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.306277 (+ 2657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.306292 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.308599 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.308605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.308859 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.308913 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.309048 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.309255 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21211,"mutex_wait_us":1,"spinlock_wait_cycles":4992,"thread_start_us":23337,"threads_started":1}
I20260811 09:06:18.310436 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.278767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55260 (local address 127.9.40.67:45293)
0811 09:06:18.287082 (+ 8315us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.287094 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.300960 (+ 13866us) server_negotiation.cc:407] Connection header received
0811 09:06:18.301322 (+ 362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.301331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.301630 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.301814 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:18.302796 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.304387 (+ 1591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.307363 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.308884 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.308984 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.309207 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.309870 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.310047 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8119,"thread_start_us":7942,"threads_started":1}
I20260811 09:06:18.310565 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.277168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53578 (local address 127.9.40.66:35793)
0811 09:06:18.298017 (+ 20849us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.298026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.298746 (+ 720us) server_negotiation.cc:407] Connection header received
0811 09:06:18.298990 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.298999 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.299221 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.299585 (+ 364us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:18.300865 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.303264 (+ 2399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.306514 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.308111 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.308240 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.308468 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.310016 (+ 1548us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.310197 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20650,"thread_start_us":20506,"threads_started":1}
I20260811 09:06:18.311335 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.278126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51704 (local address 127.9.40.65:37023)
0811 09:06:18.286526 (+ 8400us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.286537 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.297448 (+ 10911us) server_negotiation.cc:407] Connection header received
0811 09:06:18.298769 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.298781 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.299146 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.299388 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:18.301213 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.302939 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.308243 (+ 5304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.309809 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.309904 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.310129 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.310777 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.310968 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8154,"thread_start_us":7971,"threads_started":1}
I20260811 09:06:18.312036 30981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.277165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:45293 (local address 127.0.0.1:55260)
0811 09:06:18.300794 (+ 23629us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.300826 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.301077 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.302143 (+ 1066us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.302148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.302176 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:18.302580 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.302592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.305033 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.305038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.307178 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.307191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.309430 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.309437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.309637 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.309670 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.311241 (+ 1571us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.311643 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23430,"thread_start_us":19320,"threads_started":1}
I20260811 09:06:18.313071 30983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.277037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37023 (local address 127.0.0.1:51704)
0811 09:06:18.297163 (+ 20126us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.297238 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.298456 (+ 1218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.300041 (+ 1585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.300046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.300084 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:18.300990 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.301005 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.304009 (+ 3004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.304015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.308024 (+ 4009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.308055 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.310340 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.310346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.310565 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.310600 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.311545 (+ 945us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.311680 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19702,"mutex_wait_us":308,"thread_start_us":21060,"threads_started":1}
I20260811 09:06:18.315287 30725 tablet_service.cc:1467] Processing CreateTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (DEFAULT_TABLE table=test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:18.315587 30885 tablet_service.cc:1467] Processing CreateTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (DEFAULT_TABLE table=test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:18.315824 30563 tablet_service.cc:1467] Processing CreateTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (DEFAULT_TABLE table=test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:18.316982 30725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b9f4df36a54fa5900c130fe2d3e6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:18.316982 30885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b9f4df36a54fa5900c130fe2d3e6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:18.317724 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01b9f4df36a54fa5900c130fe2d3e6d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:18.330364 30987 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Bootstrap starting.
I20260811 09:06:18.332811 30989 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Bootstrap starting.
I20260811 09:06:18.334774 30988 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Bootstrap starting.
I20260811 09:06:18.336828 30987 tablet_bootstrap.cc:654] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:18.336828 30989 tablet_bootstrap.cc:654] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:18.338064 30988 tablet_bootstrap.cc:654] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:18.338310 30989 log.cc:824] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:18.338315 30987 log.cc:824] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:18.339435 30988 log.cc:824] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:18.342568 30989 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: No bootstrap required, opened a new log
I20260811 09:06:18.342639 30987 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: No bootstrap required, opened a new log
I20260811 09:06:18.342887 30989 ts_tablet_manager.cc:1397] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260811 09:06:18.342933 30987 ts_tablet_manager.cc:1397] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260811 09:06:18.344674 30988 tablet_bootstrap.cc:492] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: No bootstrap required, opened a new log
I20260811 09:06:18.345005 30988 ts_tablet_manager.cc:1397] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20260811 09:06:18.350466 30989 raft_consensus.cc:348] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.350893 30989 raft_consensus.cc:374] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:18.351016 30989 raft_consensus.cc:729] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236befc814aa4f859f2ffb3a562e3841, State: Initialized, Role: FOLLOWER
I20260811 09:06:18.351871 30989 consensus_queue.cc:260] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.354110 30964 heartbeater.cc:503] Master 127.9.40.126:33263 was elected leader, sending a full tablet report...
I20260811 09:06:18.354676 30989 ts_tablet_manager.cc:1428] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Time spent starting tablet: real 0.012s user 0.005s sys 0.006s
I20260811 09:06:18.355173 30987 raft_consensus.cc:348] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.355587 30987 raft_consensus.cc:374] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:18.355713 30987 raft_consensus.cc:729] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ca7ae58b04f46af83e686ea17d428d9, State: Initialized, Role: FOLLOWER
I20260811 09:06:18.356021 30988 raft_consensus.cc:348] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.356495 30988 raft_consensus.cc:374] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:18.356376 30987 consensus_queue.cc:260] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.356624 30988 raft_consensus.cc:729] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdeca0a713c45a78b13cee2a6575c5b, State: Initialized, Role: FOLLOWER
I20260811 09:06:18.357270 30988 consensus_queue.cc:260] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.358635 30800 heartbeater.cc:503] Master 127.9.40.126:33263 was elected leader, sending a full tablet report...
I20260811 09:06:18.359372 30988 ts_tablet_manager.cc:1428] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260811 09:06:18.361017 30987 ts_tablet_manager.cc:1428] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
W20260811 09:06:18.451831 30965 tablet.cc:2369] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:18.496106 30996 raft_consensus.cc:482] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:18.496466 30996 raft_consensus.cc:504] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.498262 30996 leader_election.cc:288] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 236befc814aa4f859f2ffb3a562e3841 (127.9.40.67:45293), 5ca7ae58b04f46af83e686ea17d428d9 (127.9.40.65:37023)
I20260811 09:06:18.508172 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.498754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57121 (local address 127.9.40.65:37023)
0811 09:06:18.498978 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.498985 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.499822 (+ 837us) server_negotiation.cc:407] Connection header received
0811 09:06:18.500021 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.500028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.500120 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.500241 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:18.501216 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.502431 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.505647 (+ 3216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.506930 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.507072 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.507190 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.507745 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.507889 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:06:18.508169 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.498826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42403 (local address 127.9.40.67:45293)
0811 09:06:18.499036 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.499044 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.499820 (+ 776us) server_negotiation.cc:407] Connection header received
0811 09:06:18.500035 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.500041 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.500120 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.500240 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:18.501216 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.502433 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.505614 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.506926 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.507072 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.507190 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.507745 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.507889 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:06:18.508390 30997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.498544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37023 (local address 127.9.40.66:57121)
0811 09:06:18.499653 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.499667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.499831 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.500303 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.500309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.500355 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:18.500922 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.500943 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.502604 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.502611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.505417 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.505435 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.507252 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.507257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.507517 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.507582 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.507694 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.507927 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":261,"spinlock_wait_cycles":5888,"thread_start_us":640,"threads_started":1}
W20260811 09:06:18.508277 30643 tablet.cc:2369] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:18.508860 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.498540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:45293 (local address 127.9.40.66:42403)
0811 09:06:18.499642 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:18.499666 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:18.499868 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:18.500303 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:18.500309 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:18.500349 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:18.501005 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.501028 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.502605 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.502611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.505417 (+ 2806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:18.505435 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.507249 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.507257 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:18.507571 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.507596 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:06:18.507694 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:06:18.507907 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":218,"spinlock_wait_cycles":40064,"thread_start_us":628,"threads_started":1}
I20260811 09:06:18.509967 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b9f4df36a54fa5900c130fe2d3e6d3" candidate_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236befc814aa4f859f2ffb3a562e3841" is_pre_election: true
I20260811 09:06:18.510236 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b9f4df36a54fa5900c130fe2d3e6d3" candidate_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ca7ae58b04f46af83e686ea17d428d9" is_pre_election: true
I20260811 09:06:18.510491 30910 raft_consensus.cc:2454] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdeca0a713c45a78b13cee2a6575c5b in term 0.
I20260811 09:06:18.510756 30586 raft_consensus.cc:2454] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cdeca0a713c45a78b13cee2a6575c5b in term 0.
I20260811 09:06:18.511251 30671 leader_election.cc:302] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236befc814aa4f859f2ffb3a562e3841, 5cdeca0a713c45a78b13cee2a6575c5b; no voters:
I20260811 09:06:18.511688 30996 raft_consensus.cc:2781] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:18.511868 30996 raft_consensus.cc:482] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:18.511982 30996 raft_consensus.cc:3037] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:18.514369 30996 raft_consensus.cc:504] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.515049 30996 leader_election.cc:288] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [CANDIDATE]: Term 1 election: Requested vote from peers 236befc814aa4f859f2ffb3a562e3841 (127.9.40.67:45293), 5ca7ae58b04f46af83e686ea17d428d9 (127.9.40.65:37023)
I20260811 09:06:18.515622 30910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b9f4df36a54fa5900c130fe2d3e6d3" candidate_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236befc814aa4f859f2ffb3a562e3841"
I20260811 09:06:18.515662 30586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01b9f4df36a54fa5900c130fe2d3e6d3" candidate_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ca7ae58b04f46af83e686ea17d428d9"
I20260811 09:06:18.515831 30910 raft_consensus.cc:3037] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:18.515919 30586 raft_consensus.cc:3037] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:18.518312 30910 raft_consensus.cc:2454] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdeca0a713c45a78b13cee2a6575c5b in term 1.
I20260811 09:06:18.518332 30586 raft_consensus.cc:2454] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cdeca0a713c45a78b13cee2a6575c5b in term 1.
I20260811 09:06:18.518823 30671 leader_election.cc:302] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236befc814aa4f859f2ffb3a562e3841, 5cdeca0a713c45a78b13cee2a6575c5b; no voters:
I20260811 09:06:18.519200 30996 raft_consensus.cc:2781] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:18.519771 30996 raft_consensus.cc:686] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 1 LEADER]: Becoming Leader. State: Replica: 5cdeca0a713c45a78b13cee2a6575c5b, State: Running, Role: LEADER
I20260811 09:06:18.520380 30996 consensus_queue.cc:237] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } }
I20260811 09:06:18.525277 30415 catalog_manager.cc:5495] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cdeca0a713c45a78b13cee2a6575c5b (127.9.40.66). New cstate: current_term: 1 leader_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cdeca0a713c45a78b13cee2a6575c5b" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 } health_report { overall_health: UNKNOWN } } }
I20260811 09:06:18.554808 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.549098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53580 (local address 127.9.40.66:35793)
0811 09:06:18.549320 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.549325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.549344 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:06:18.549433 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.549436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.549531 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.549668 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:18.550100 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.550861 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.552263 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.552867 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.553003 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.553574 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:18.554124 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:18.554270 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.554509 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.554557 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:06:18.563773 30910 raft_consensus.cc:1261] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 1 FOLLOWER]: Refusing update from remote peer 5cdeca0a713c45a78b13cee2a6575c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:18.563772 30586 raft_consensus.cc:1261] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 1 FOLLOWER]: Refusing update from remote peer 5cdeca0a713c45a78b13cee2a6575c5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:18.564962 30999 consensus_queue.cc:1035] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "236befc814aa4f859f2ffb3a562e3841" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 45293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:18.565393 30996 consensus_queue.cc:1035] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ca7ae58b04f46af83e686ea17d428d9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:18.576171 31002 mvcc.cc:205] Tried to move back new op lower bound from 7317254875386601472 to 7317254875223912448. Current Snapshot: MvccSnapshot[applied={T|T < 7317254875386601472}]
I20260811 09:06:18.576802 31005 mvcc.cc:205] Tried to move back new op lower bound from 7317254875386601472 to 7317254875223912448. Current Snapshot: MvccSnapshot[applied={T|T < 7317254875386601472}]
I20260811 09:06:18.597716 30414 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:18.598048 30414 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:18.602090 30414 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 35517a069ade484788d732706fa67b54: Sending DeleteTablet for 3 replicas of tablet 01b9f4df36a54fa5900c130fe2d3e6d3
I20260811 09:06:18.603421 30725 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:18 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53578
I20260811 09:06:18.603403 30885 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:18 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:55260
I20260811 09:06:18.604239 30563 tablet_service.cc:1514] Processing DeleteTablet for tablet 01b9f4df36a54fa5900c130fe2d3e6d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:18 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51704
I20260811 09:06:18.604524 31012 tablet_replica.cc:330] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: stopping tablet replica
I20260811 09:06:18.605094 31012 raft_consensus.cc:2229] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:18.606070 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30490
I20260811 09:06:18.606055 31012 raft_consensus.cc:2258] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:18.608525 31012 ts_tablet_manager.cc:1905] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:18.618438 31012 ts_tablet_manager.cc:1918] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:18.618832 31012 log.cc:1196] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/01b9f4df36a54fa5900c130fe2d3e6d3
I20260811 09:06:18.619357 31013 tablet_replica.cc:330] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: stopping tablet replica
I20260811 09:06:18.619786 31012 ts_tablet_manager.cc:1939] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5cdeca0a713c45a78b13cee2a6575c5b: Deleting consensus metadata
I20260811 09:06:18.619891 31013 raft_consensus.cc:2229] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:18.620476 31013 raft_consensus.cc:2258] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:18.622345 30399 catalog_manager.cc:4841] TS 5cdeca0a713c45a78b13cee2a6575c5b (127.9.40.66:35793): tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (table test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]) successfully deleted
I20260811 09:06:18.623023 31013 ts_tablet_manager.cc:1905] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:18.626147 31014 tablet_replica.cc:330] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: stopping tablet replica
I20260811 09:06:18.626557 31014 raft_consensus.cc:2229] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:18.627035 31014 raft_consensus.cc:2258] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:18.628892 31014 ts_tablet_manager.cc:1905] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:18.632817 31013 ts_tablet_manager.cc:1918] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:18.632999 31013 log.cc:1196] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/01b9f4df36a54fa5900c130fe2d3e6d3
I20260811 09:06:18.633522 31013 ts_tablet_manager.cc:1939] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 236befc814aa4f859f2ffb3a562e3841: Deleting consensus metadata
I20260811 09:06:18.635577 30398 catalog_manager.cc:4841] TS 236befc814aa4f859f2ffb3a562e3841 (127.9.40.67:45293): tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (table test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]) successfully deleted
I20260811 09:06:18.636479 31014 ts_tablet_manager.cc:1918] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:18.636683 31014 log.cc:1196] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/01b9f4df36a54fa5900c130fe2d3e6d3
I20260811 09:06:18.637321 31014 ts_tablet_manager.cc:1939] T 01b9f4df36a54fa5900c130fe2d3e6d3 P 5ca7ae58b04f46af83e686ea17d428d9: Deleting consensus metadata
Aug 11 09:06:18 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:06:18.639518 30400 catalog_manager.cc:4841] TS 5ca7ae58b04f46af83e686ea17d428d9 (127.9.40.65:37023): tablet 01b9f4df36a54fa5900c130fe2d3e6d3 (table test-table [id=7d6b9d55f09b410db7f7dc4db1fd5166]) successfully deleted
I20260811 09:06:18.646723 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.608919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51712 (local address 127.9.40.65:37023)
0811 09:06:18.609514 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.609521 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.625378 (+ 15857us) server_negotiation.cc:407] Connection header received
0811 09:06:18.625611 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.625618 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.625751 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.625880 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:18.628905 (+ 3025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.629998 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.632067 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.632781 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.632911 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.639275 (+ 6364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:18.639292 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:18.639312 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:18.639366 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:18.643495 (+ 4129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.644275 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.644285 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.644289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.644464 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.644923 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.644928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.644932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.645427 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:18.645748 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.646069 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.646311 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452}
I20260811 09:06:18.647450 30636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:18.743757 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30490
I20260811 09:06:18.770254 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30649
Aug 11 09:06:18 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:06:18.789072 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.771298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53582 (local address 127.9.40.66:35793)
0811 09:06:18.771476 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.771523 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.771541 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:06:18.771598 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.771601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.771674 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.771760 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:18.773585 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.774427 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.776351 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.776891 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.777000 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.782439 (+ 5439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:18.782454 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:18.782472 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:18.782547 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:18.786027 (+ 3480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.786745 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.786754 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.786756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.786906 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.787301 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.787306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.787316 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.787729 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:18.788114 (+ 385us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.788419 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.788668 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:06:18.789913 30795 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:18.909314 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30649
I20260811 09:06:18.934360 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30808
Aug 11 09:06:18 dist-test-slave-cbsf krb5kdc[30364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:06:18.951087 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:18.935118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.9.40.67:45293)
0811 09:06:18.935292 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:18.935296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:18.935365 (+ 69us) server_negotiation.cc:407] Connection header received
0811 09:06:18.935576 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:18.935581 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:18.935662 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:18.935772 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:18.937124 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.937932 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.939333 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:18.939791 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:18.939889 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:18.944871 (+ 4982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:18.944902 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:18.944920 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:18.944967 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:18.947867 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.948766 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.948775 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.948777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.948937 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:18.949359 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:18.949364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:18.949373 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:18.949774 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:18.950120 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:18.950438 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:06:18.950679 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:06:18.951848 30957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:19.053431 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30808
I20260811 09:06:19.076138 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 30380
I20260811 09:06:19.077301 30453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:19.193811 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 30380
2026-08-11T09:06:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4970 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T09:06:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:19Z Disabled control of system clock
I20260811 09:06:19.300318 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44263
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:19.663326 31028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:19.663679 31028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:19.663837 31028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:19.671504 31028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:19.671638 31028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:19.671715 31028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:19.671823 31028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:19.671887 31028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:19.681610 31028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44263
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:19.683185 31028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:19.685570 31028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:19.698415 31033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:19.698370 31034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:19.698401 31036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:19.699111 31028 server_base.cc:1034] running on GCE node
I20260811 09:06:19.699999 31028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:19.701563 31028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:19.702775 31028 hybrid_clock.cc:648] HybridClock initialized: now 1786439179702708 us; error 57 us; skew 500 ppm
I20260811 09:06:19.703181 31028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:19.705734 31028 webserver.cc:460] Webserver started at http://127.9.40.126:45615/ using document root <none> and password file <none>
I20260811 09:06:19.706499 31028 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:19.706601 31028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:19.706933 31028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:19.710186 31028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "2b5817e7baf44fb485256401962519c4"
format_stamp: "Formatted at 2026-08-11 09:06:19 on dist-test-slave-cbsf"
server_key: "fae8f575c7c841ee9760c8dee84d1ea1"
server_key_iv: "0affcb0b11c5cf28b0b2eefd922459e0"
server_key_version: "encryptionkey@0"
I20260811 09:06:19.711076 31028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2b5817e7baf44fb485256401962519c4"
format_stamp: "Formatted at 2026-08-11 09:06:19 on dist-test-slave-cbsf"
server_key: "fae8f575c7c841ee9760c8dee84d1ea1"
server_key_iv: "0affcb0b11c5cf28b0b2eefd922459e0"
server_key_version: "encryptionkey@0"
I20260811 09:06:19.716823 31028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:06:19.720769 31044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:19.722285 31028 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:06:19.722568 31028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2b5817e7baf44fb485256401962519c4"
format_stamp: "Formatted at 2026-08-11 09:06:19 on dist-test-slave-cbsf"
server_key: "fae8f575c7c841ee9760c8dee84d1ea1"
server_key_iv: "0affcb0b11c5cf28b0b2eefd922459e0"
server_key_version: "encryptionkey@0"
I20260811 09:06:19.722779 31028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:19.745072 31028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:19.746026 31028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:19.746354 31028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:19.766686 31028 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44263
I20260811 09:06:19.766731 31105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44263 every 8 connection(s)
I20260811 09:06:19.768173 31028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:19.772531 31106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:19.773135 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31028
I20260811 09:06:19.773321 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:19.773981 9377 external_mini_cluster.cc:1428] Setting key d0c2df5fede26bc4bd4ae2f4c267348b
I20260811 09:06:19.787315 31106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: Bootstrap starting.
I20260811 09:06:19.789103 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:19.776015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45334 (local address 127.9.40.126:44263)
0811 09:06:19.777268 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:19.777280 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:19.777323 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:06:19.779335 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:19.779359 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:19.779838 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:19.780194 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:19.780831 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:19.782177 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:19.784245 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:19.785084 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:19.785225 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:19.786568 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:19.786577 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:19.786592 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:19.787181 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:19.787299 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:19.787334 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:19.787474 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:19.787726 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 09:06:19.788055 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":527,"threads_started":1}
I20260811 09:06:19.791121 31106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:19.792493 31106 log.cc:824] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:19.795401 31106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: No bootstrap required, opened a new log
I20260811 09:06:19.803956 31106 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } }
I20260811 09:06:19.804385 31106 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:19.804508 31106 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b5817e7baf44fb485256401962519c4, State: Initialized, Role: FOLLOWER
I20260811 09:06:19.805119 31106 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } }
I20260811 09:06:19.805364 31106 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:19.805563 31106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:19.805727 31106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:19.808095 31106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } }
I20260811 09:06:19.808636 31106 leader_election.cc:302] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b5817e7baf44fb485256401962519c4; no voters:
I20260811 09:06:19.809401 31106 leader_election.cc:288] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:19.809569 31112 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:19.810437 31112 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [term 1 LEADER]: Becoming Leader. State: Replica: 2b5817e7baf44fb485256401962519c4, State: Running, Role: LEADER
I20260811 09:06:19.811010 31112 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } }
I20260811 09:06:19.811646 31106 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:19.814350 31113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b5817e7baf44fb485256401962519c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } } }
I20260811 09:06:19.814880 31113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:19.814340 31114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b5817e7baf44fb485256401962519c4. Latest consensus state: current_term: 1 leader_uuid: "2b5817e7baf44fb485256401962519c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b5817e7baf44fb485256401962519c4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44263 } } }
I20260811 09:06:19.816043 31114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:19.816951 31119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:19.823451 31119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:19.832490 31119 catalog_manager.cc:1347] Generated new cluster ID: 7d5a2c267cc0459d9f11c4cbda0ff95b
I20260811 09:06:19.832633 31119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:19.846563 31119 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:19.847985 31119 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:19.858834 31119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: Generated new TSK 0
I20260811 09:06:19.859755 31119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:19.867350 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--builtin_ntp_servers=127.9.40.84:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:20.231096 31137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:20.231436 31137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:20.231599 31137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:20.239094 31137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:20.239250 31137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:20.239430 31137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:20.249024 31137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:20.250579 31137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:20.252497 31137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:20.266188 31147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:20.266120 31145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:20.266227 31144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:20.267189 31137 server_base.cc:1034] running on GCE node
I20260811 09:06:20.267867 31137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:20.268991 31137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:20.270227 31137 hybrid_clock.cc:648] HybridClock initialized: now 1786439180270162 us; error 63 us; skew 500 ppm
I20260811 09:06:20.270579 31137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:20.273067 31137 webserver.cc:460] Webserver started at http://127.9.40.65:37361/ using document root <none> and password file <none>
I20260811 09:06:20.273855 31137 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:20.273989 31137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:20.274331 31137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:20.277467 31137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "9d3230fb3801425abe19cc437b195811"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "2a7d45de10264b7b100cdb4bde765164"
server_key_iv: "31e2e01114d9f8b364416cf9b9d56cdd"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.278311 31137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "9d3230fb3801425abe19cc437b195811"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "2a7d45de10264b7b100cdb4bde765164"
server_key_iv: "31e2e01114d9f8b364416cf9b9d56cdd"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.283869 31137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:06:20.288026 31155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.290128 31137 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:06:20.290390 31137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "9d3230fb3801425abe19cc437b195811"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "2a7d45de10264b7b100cdb4bde765164"
server_key_iv: "31e2e01114d9f8b364416cf9b9d56cdd"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.290740 31137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:20.304865 31137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:20.305685 31137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:20.305931 31137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:20.306941 31137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:20.308480 31137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:20.308570 31137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.308656 31137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:20.308770 31137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.343489 31137 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38099
I20260811 09:06:20.343547 31282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38099 every 8 connection(s)
I20260811 09:06:20.345037 31137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:20.353760 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31137
I20260811 09:06:20.353946 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:20.354434 9377 external_mini_cluster.cc:1428] Setting key 00576ff43a0c61513a26f161f45c7b4e
I20260811 09:06:20.357672 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--builtin_ntp_servers=127.9.40.84:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:20.358407 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:20.347673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42513 (local address 127.9.40.126:44263)
0811 09:06:20.348184 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:20.348190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:20.349353 (+ 1163us) server_negotiation.cc:407] Connection header received
0811 09:06:20.351121 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:20.351125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:20.351197 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:20.351297 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:20.352336 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.353155 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.354932 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.355494 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.355630 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:20.357356 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:20.357365 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:20.357373 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:20.357400 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:20.357457 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:20.357478 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:20.357573 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:20.357936 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 09:06:20.358040 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":235,"threads_started":1}
I20260811 09:06:20.359639 31291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:20.348215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44263 (local address 127.9.40.65:42513)
0811 09:06:20.349198 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:20.349241 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:20.350862 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:20.351497 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:20.351506 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:20.351595 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:20.352185 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:20.352202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.353294 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.353298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:20.354773 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:20.354786 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.355743 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.355751 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:20.356061 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:20.356971 (+ 910us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:20.356974 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:20.357023 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:20.357126 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:20.357593 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:20.357602 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:20.357627 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:06:20.357844 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 09:06:20.358309 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":295,"threads_started":1}
I20260811 09:06:20.361325 31284 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44263
I20260811 09:06:20.361670 31284 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:20.362430 31284 heartbeater.cc:511] Master 127.9.40.126:44263 requested a full tablet report, sending...
I20260811 09:06:20.364542 31064 ts_manager.cc:194] Registered new tserver with Master: 9d3230fb3801425abe19cc437b195811 (127.9.40.65:38099)
I20260811 09:06:20.366062 31064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:42513
W20260811 09:06:20.728287 31292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:20.728634 31292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:20.728812 31292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:20.736352 31292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:20.736469 31292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:20.736644 31292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:20.746235 31292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:20.747670 31292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:20.749589 31292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:20.764374 31297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:20.764343 31301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:20.764420 31298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:20.765452 31292 server_base.cc:1034] running on GCE node
I20260811 09:06:20.766302 31292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:20.767571 31292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:20.768805 31292 hybrid_clock.cc:648] HybridClock initialized: now 1786439180768738 us; error 59 us; skew 500 ppm
I20260811 09:06:20.769214 31292 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:20.772140 31292 webserver.cc:460] Webserver started at http://127.9.40.66:43595/ using document root <none> and password file <none>
I20260811 09:06:20.773097 31292 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:20.773213 31292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:20.773592 31292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:20.777521 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "fa580d02c214482d80160ddbb37536b0"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "d04318c0c5407ceb29439270c301874c"
server_key_iv: "4a9a8ef9df3d9167b12d58b27ce67b3c"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.778514 31292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "fa580d02c214482d80160ddbb37536b0"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "d04318c0c5407ceb29439270c301874c"
server_key_iv: "4a9a8ef9df3d9167b12d58b27ce67b3c"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.785740 31292 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260811 09:06:20.790117 31309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.791471 31292 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:20.791673 31292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "fa580d02c214482d80160ddbb37536b0"
format_stamp: "Formatted at 2026-08-11 09:06:20 on dist-test-slave-cbsf"
server_key: "d04318c0c5407ceb29439270c301874c"
server_key_iv: "4a9a8ef9df3d9167b12d58b27ce67b3c"
server_key_version: "encryptionkey@0"
I20260811 09:06:20.791872 31292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:20.812850 31292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:20.813772 31292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:20.814031 31292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:20.815021 31292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:20.816591 31292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:20.816730 31292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.816855 31292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:20.816942 31292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:20.852839 31292 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46867
I20260811 09:06:20.852929 31452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46867 every 8 connection(s)
I20260811 09:06:20.854235 31292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:20.864677 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31292
I20260811 09:06:20.864884 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:20.865494 9377 external_mini_cluster.cc:1428] Setting key fa6932eaef6a56c10369b85ae92bad66
I20260811 09:06:20.867689 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:20.857078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56417 (local address 127.9.40.126:44263)
0811 09:06:20.857281 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:20.857287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:20.858771 (+ 1484us) server_negotiation.cc:407] Connection header received
0811 09:06:20.860838 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:20.860842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:20.860910 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:20.861000 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:20.862152 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.862854 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.864512 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.865174 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.865306 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:20.866862 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:20.866867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:20.866870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:20.866887 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:20.866920 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:20.866932 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:20.867002 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:20.867257 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 09:06:20.867391 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:06:20.868571 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--builtin_ntp_servers=127.9.40.84:37897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:20.868973 31455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:20.857582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44263 (local address 127.9.40.66:56417)
0811 09:06:20.858611 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:20.858648 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:20.860534 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:20.861284 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:20.861292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:20.861380 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:20.861999 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:20.862016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.863019 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.863023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:20.864354 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:20.864367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:20.865405 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:20.865418 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:20.865692 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:20.866481 (+ 789us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:20.866485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:20.866559 (+ 74us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:20.866676 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:20.867026 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:20.867038 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:20.867046 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:06:20.867252 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 09:06:20.867740 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":336,"threads_started":1}
I20260811 09:06:20.870481 31453 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44263
I20260811 09:06:20.870841 31453 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:20.871819 31453 heartbeater.cc:511] Master 127.9.40.126:44263 requested a full tablet report, sending...
I20260811 09:06:20.873634 31064 ts_manager.cc:194] Registered new tserver with Master: fa580d02c214482d80160ddbb37536b0 (127.9.40.66:46867)
I20260811 09:06:20.874482 31064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:56417
W20260811 09:06:21.224254 31456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:21.224563 31456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:21.224771 31456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:21.232299 31456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:21.232406 31456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:21.232581 31456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:21.242045 31456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44263
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:21.243485 31456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:21.245589 31456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:21.259610 31461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:21.259580 31464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:21.259550 31462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:21.260375 31456 server_base.cc:1034] running on GCE node
I20260811 09:06:21.261149 31456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:21.262173 31456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:21.263444 31456 hybrid_clock.cc:648] HybridClock initialized: now 1786439181263389 us; error 40 us; skew 500 ppm
I20260811 09:06:21.263851 31456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:21.266294 31456 webserver.cc:460] Webserver started at http://127.9.40.67:39053/ using document root <none> and password file <none>
I20260811 09:06:21.267023 31456 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:21.267158 31456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:21.267498 31456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:21.270587 31456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a"
format_stamp: "Formatted at 2026-08-11 09:06:21 on dist-test-slave-cbsf"
server_key: "194b721da28b4107efcb730ae87a2d23"
server_key_iv: "fe2c5c056007963d3cb51a4ef7d83cad"
server_key_version: "encryptionkey@0"
I20260811 09:06:21.271452 31456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a"
format_stamp: "Formatted at 2026-08-11 09:06:21 on dist-test-slave-cbsf"
server_key: "194b721da28b4107efcb730ae87a2d23"
server_key_iv: "fe2c5c056007963d3cb51a4ef7d83cad"
server_key_version: "encryptionkey@0"
I20260811 09:06:21.276908 31456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 09:06:21.280443 31471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:21.281823 31456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:21.282037 31456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a"
format_stamp: "Formatted at 2026-08-11 09:06:21 on dist-test-slave-cbsf"
server_key: "194b721da28b4107efcb730ae87a2d23"
server_key_iv: "fe2c5c056007963d3cb51a4ef7d83cad"
server_key_version: "encryptionkey@0"
I20260811 09:06:21.282274 31456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:21.300809 31456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:21.301657 31456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:21.301949 31456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:21.302986 31456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:21.304494 31456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:21.304620 31456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:21.304764 31456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:21.304852 31456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:21.339640 31456 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44837
I20260811 09:06:21.339732 31607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44837 every 8 connection(s)
I20260811 09:06:21.341094 31456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:21.344044 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31456
I20260811 09:06:21.344265 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:21.344857 9377 external_mini_cluster.cc:1428] Setting key 3361583788a16b2dc5e15920c2500709
I20260811 09:06:21.354377 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.344306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54351 (local address 127.9.40.126:44263)
0811 09:06:21.344517 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.344525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.346059 (+ 1534us) server_negotiation.cc:407] Connection header received
0811 09:06:21.348325 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.348330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.348416 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.348533 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:21.349600 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.350342 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.351855 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.352266 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.352360 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.353702 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:21.353707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:21.353711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:21.353729 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:21.353758 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:21.353770 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:21.353840 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.354043 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.354123 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:06:21.355356 31610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.344588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44263 (local address 127.9.40.67:54351)
0811 09:06:21.345891 (+ 1303us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.345942 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.348031 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.348805 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.348812 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.348895 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:21.349446 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.349463 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.350473 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.350477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.351729 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.351740 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.352445 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.352449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.352660 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.353385 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:21.353388 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:21.353433 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:21.353511 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:21.353853 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:21.353861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:21.353867 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.354001 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.354286 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":440,"threads_started":1}
I20260811 09:06:21.356846 31608 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44263
I20260811 09:06:21.357178 31608 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:21.357959 31608 heartbeater.cc:511] Master 127.9.40.126:44263 requested a full tablet report, sending...
I20260811 09:06:21.359539 31064 ts_manager.cc:194] Registered new tserver with Master: e927a202ebbb4ae4bbd4d9bae3d7e18a (127.9.40.67:44837)
I20260811 09:06:21.360443 31064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:54351
I20260811 09:06:21.360509 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:21.369537 31284 heartbeater.cc:503] Master 127.9.40.126:44263 was elected leader, sending a full tablet report...
I20260811 09:06:21.370903 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.364645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42152 (local address 127.9.40.126:44263)
0811 09:06:21.364943 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.364952 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.365293 (+ 341us) server_negotiation.cc:407] Connection header received
0811 09:06:21.365455 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.365460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.365522 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.365625 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:21.366067 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.366775 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.368153 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.368557 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.368659 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.369984 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:21.369989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:21.369992 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:21.370011 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:21.370058 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:21.370076 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:21.370254 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.370448 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.370571 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 09:06:21.374562 31064 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42152:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:21.381435 31064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:21.412945 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.394149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46867 (local address 127.0.0.1:37164)
0811 09:06:21.402863 (+ 8714us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.402917 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.403177 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.404347 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.404358 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.404419 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:21.405585 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.405595 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.407493 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.407499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.409546 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.409560 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.411773 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.411779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.411994 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.412055 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.412197 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.412454 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8205,"mutex_wait_us":638,"spinlock_wait_cycles":17280,"thread_start_us":8401,"threads_started":1}
I20260811 09:06:21.412920 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.394173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37164 (local address 127.9.40.66:46867)
0811 09:06:21.401982 (+ 7809us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.401993 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.403025 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 09:06:21.403458 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.403466 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.403740 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.403940 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:21.405809 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.407331 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.409761 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.411260 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.411389 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.411629 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.412265 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.412475 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7578,"thread_start_us":7424,"threads_started":1}
I20260811 09:06:21.413733 31626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.403372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44837 (local address 127.0.0.1:50036)
0811 09:06:21.405406 (+ 2034us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.405431 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.405580 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.406441 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.406446 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.406469 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:21.406853 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.406863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.408822 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.408827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.410993 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.411010 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.412817 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.412825 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.413073 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.413106 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.413193 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.413332 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1888,"mutex_wait_us":1,"thread_start_us":1660,"threads_started":1}
I20260811 09:06:21.413959 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.394867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.9.40.65:38099)
0811 09:06:21.395635 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.395648 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.403031 (+ 7383us) server_negotiation.cc:407] Connection header received
0811 09:06:21.403448 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.403458 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.403787 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.403983 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:21.405421 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.407080 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.409430 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.410517 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.410688 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.411012 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.413350 (+ 2338us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.413542 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":330,"threads_started":1}
I20260811 09:06:21.414371 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.403502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.9.40.67:44837)
0811 09:06:21.404998 (+ 1496us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.405009 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.405550 (+ 541us) server_negotiation.cc:407] Connection header received
0811 09:06:21.405727 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.405736 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.405988 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.406178 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:21.407077 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.408667 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.411234 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.412597 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.412767 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.413038 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.413904 (+ 866us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.414055 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1312,"thread_start_us":1178,"threads_started":1}
I20260811 09:06:21.413771 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.394180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38099 (local address 127.0.0.1:41870)
0811 09:06:21.402866 (+ 8686us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.402916 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.403180 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.404347 (+ 1167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.404363 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.404425 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:21.405186 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.405199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.407245 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.407250 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.409199 (+ 1949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.409214 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.410727 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.410733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.413169 (+ 2436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.413204 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.413271 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.413396 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8550,"mutex_wait_us":2180,"spinlock_wait_cycles":43264,"thread_start_us":8042,"threads_started":1}
I20260811 09:06:21.418040 31531 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 (DEFAULT_TABLE table=test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:21.418041 31370 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 (DEFAULT_TABLE table=test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:21.418041 31208 tablet_service.cc:1467] Processing CreateTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 (DEFAULT_TABLE table=test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:21.419543 31531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf1b991a6394dd0864310a02dca74a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:21.419553 31208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf1b991a6394dd0864310a02dca74a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:21.419554 31370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cf1b991a6394dd0864310a02dca74a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:21.431799 31628 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Bootstrap starting.
I20260811 09:06:21.434494 31632 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Bootstrap starting.
I20260811 09:06:21.435396 31630 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Bootstrap starting.
I20260811 09:06:21.437659 31628 tablet_bootstrap.cc:654] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:21.438723 31632 tablet_bootstrap.cc:654] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:21.439067 31628 log.cc:824] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:21.439167 31630 tablet_bootstrap.cc:654] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:21.439765 31632 log.cc:824] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:21.440275 31630 log.cc:824] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:21.442657 31628 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: No bootstrap required, opened a new log
I20260811 09:06:21.442992 31628 ts_tablet_manager.cc:1397] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 09:06:21.443193 31630 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: No bootstrap required, opened a new log
I20260811 09:06:21.443462 31630 ts_tablet_manager.cc:1397] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:06:21.444880 31632 tablet_bootstrap.cc:492] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: No bootstrap required, opened a new log
I20260811 09:06:21.445168 31632 ts_tablet_manager.cc:1397] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.001s
I20260811 09:06:21.452495 31632 raft_consensus.cc:348] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.452986 31632 raft_consensus.cc:374] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:21.453163 31632 raft_consensus.cc:729] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa580d02c214482d80160ddbb37536b0, State: Initialized, Role: FOLLOWER
I20260811 09:06:21.454103 31632 consensus_queue.cc:260] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.454325 31630 raft_consensus.cc:348] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.454879 31630 raft_consensus.cc:374] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:21.455039 31630 raft_consensus.cc:729] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d3230fb3801425abe19cc437b195811, State: Initialized, Role: FOLLOWER
I20260811 09:06:21.455034 31628 raft_consensus.cc:348] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.455327 31453 heartbeater.cc:503] Master 127.9.40.126:44263 was elected leader, sending a full tablet report...
I20260811 09:06:21.455529 31628 raft_consensus.cc:374] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:21.455680 31628 raft_consensus.cc:729] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e927a202ebbb4ae4bbd4d9bae3d7e18a, State: Initialized, Role: FOLLOWER
I20260811 09:06:21.456012 31632 ts_tablet_manager.cc:1428] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Time spent starting tablet: real 0.011s user 0.007s sys 0.002s
I20260811 09:06:21.455842 31630 consensus_queue.cc:260] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.456470 31628 consensus_queue.cc:260] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.457921 31630 ts_tablet_manager.cc:1428] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 09:06:21.459859 31608 heartbeater.cc:503] Master 127.9.40.126:44263 was elected leader, sending a full tablet report...
I20260811 09:06:21.460518 31628 ts_tablet_manager.cc:1428] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Time spent starting tablet: real 0.017s user 0.015s sys 0.000s
I20260811 09:06:21.479019 31637 raft_consensus.cc:482] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:21.479383 31637 raft_consensus.cc:504] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.481133 31637 leader_election.cc:288] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa580d02c214482d80160ddbb37536b0 (127.9.40.66:46867), e927a202ebbb4ae4bbd4d9bae3d7e18a (127.9.40.67:44837)
I20260811 09:06:21.487514 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.481651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53033 (local address 127.9.40.66:46867)
0811 09:06:21.481897 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.481905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.482112 (+ 207us) server_negotiation.cc:407] Connection header received
0811 09:06:21.482359 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.482369 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.482493 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.482593 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:21.483192 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.483886 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.485805 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.486552 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.486717 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.486833 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.487101 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.487227 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:06:21.487555 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.481365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46867 (local address 127.9.40.65:53033)
0811 09:06:21.481982 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.481998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.482178 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.482649 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.482653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.482683 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:21.483047 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.483056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.484026 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.484031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.485597 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.485609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.486768 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.486774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.486941 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.486965 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.487074 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.487208 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":343,"threads_started":1}
I20260811 09:06:21.488124 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.482425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43919 (local address 127.9.40.67:44837)
0811 09:06:21.482628 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.482633 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.482832 (+ 199us) server_negotiation.cc:407] Connection header received
0811 09:06:21.483030 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.483035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.483104 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.483196 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:21.483703 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.484401 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.486278 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.487208 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.487351 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.487455 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.487806 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.487935 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:06:21.488883 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.482160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44837 (local address 127.9.40.65:43919)
0811 09:06:21.482737 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:21.482753 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:21.482898 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:21.483211 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:21.483215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:21.483239 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:21.483553 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.483560 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.484531 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.484536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.486102 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:21.486113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.487399 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.487405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:21.487580 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.487643 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 09:06:21.488368 (+ 725us) client_negotiation.cc:240] Negotiation successful
0811 09:06:21.488520 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":271,"threads_started":1}
I20260811 09:06:21.488665 31393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf1b991a6394dd0864310a02dca74a0" candidate_uuid: "9d3230fb3801425abe19cc437b195811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa580d02c214482d80160ddbb37536b0" is_pre_election: true
I20260811 09:06:21.489318 31393 raft_consensus.cc:2454] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3230fb3801425abe19cc437b195811 in term 0.
I20260811 09:06:21.490034 31158 leader_election.cc:302] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3230fb3801425abe19cc437b195811, fa580d02c214482d80160ddbb37536b0; no voters:
I20260811 09:06:21.489910 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf1b991a6394dd0864310a02dca74a0" candidate_uuid: "9d3230fb3801425abe19cc437b195811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" is_pre_election: true
I20260811 09:06:21.490461 31553 raft_consensus.cc:2454] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d3230fb3801425abe19cc437b195811 in term 0.
I20260811 09:06:21.490473 31637 raft_consensus.cc:2781] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:21.490639 31637 raft_consensus.cc:482] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:21.490880 31637 raft_consensus.cc:3037] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:21.493281 31637 raft_consensus.cc:504] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.493907 31637 leader_election.cc:288] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [CANDIDATE]: Term 1 election: Requested vote from peers fa580d02c214482d80160ddbb37536b0 (127.9.40.66:46867), e927a202ebbb4ae4bbd4d9bae3d7e18a (127.9.40.67:44837)
I20260811 09:06:21.494417 31393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf1b991a6394dd0864310a02dca74a0" candidate_uuid: "9d3230fb3801425abe19cc437b195811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa580d02c214482d80160ddbb37536b0"
I20260811 09:06:21.494483 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cf1b991a6394dd0864310a02dca74a0" candidate_uuid: "9d3230fb3801425abe19cc437b195811" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a"
I20260811 09:06:21.494632 31393 raft_consensus.cc:3037] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:21.494737 31553 raft_consensus.cc:3037] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:21.497849 31553 raft_consensus.cc:2454] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3230fb3801425abe19cc437b195811 in term 1.
I20260811 09:06:21.497881 31393 raft_consensus.cc:2454] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d3230fb3801425abe19cc437b195811 in term 1.
I20260811 09:06:21.498268 31158 leader_election.cc:302] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d3230fb3801425abe19cc437b195811, e927a202ebbb4ae4bbd4d9bae3d7e18a; no voters:
I20260811 09:06:21.498629 31637 raft_consensus.cc:2781] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:21.499075 31637 raft_consensus.cc:686] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 1 LEADER]: Becoming Leader. State: Replica: 9d3230fb3801425abe19cc437b195811, State: Running, Role: LEADER
I20260811 09:06:21.499588 31637 consensus_queue.cc:237] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } }
I20260811 09:06:21.504171 31063 catalog_manager.cc:5495] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d3230fb3801425abe19cc437b195811 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "9d3230fb3801425abe19cc437b195811" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d3230fb3801425abe19cc437b195811" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 } health_report { overall_health: UNKNOWN } } }
I20260811 09:06:21.534552 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.528947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41874 (local address 127.9.40.65:38099)
0811 09:06:21.529199 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.529204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.529224 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:06:21.529399 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.529403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.529539 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.529674 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:21.530041 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.530786 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.532143 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.532637 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.532846 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.533342 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:21.533892 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:21.534030 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.534137 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.534177 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:06:21.543093 31553 raft_consensus.cc:1261] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 1 FOLLOWER]: Refusing update from remote peer 9d3230fb3801425abe19cc437b195811: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:21.543136 31393 raft_consensus.cc:1261] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 1 FOLLOWER]: Refusing update from remote peer 9d3230fb3801425abe19cc437b195811: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:21.544104 31643 consensus_queue.cc:1035] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e927a202ebbb4ae4bbd4d9bae3d7e18a" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:21.544530 31637 consensus_queue.cc:1035] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa580d02c214482d80160ddbb37536b0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:21.572598 31064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:21.572961 31064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42152:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:21.576946 31064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b5817e7baf44fb485256401962519c4: Sending DeleteTablet for 3 replicas of tablet 2cf1b991a6394dd0864310a02dca74a0
I20260811 09:06:21.578274 31208 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:21 UTC) from {username='slave'} at 127.0.0.1:41870
I20260811 09:06:21.578619 31370 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:21 UTC) from {username='slave'} at 127.0.0.1:37164
I20260811 09:06:21.579357 31531 tablet_service.cc:1514] Processing DeleteTablet for tablet 2cf1b991a6394dd0864310a02dca74a0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:21 UTC) from {username='slave'} at 127.0.0.1:50036
I20260811 09:06:21.579577 31661 tablet_replica.cc:330] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: stopping tablet replica
I20260811 09:06:21.579799 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31137
I20260811 09:06:21.580025 31661 raft_consensus.cc:2229] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:21.581553 31661 raft_consensus.cc:2258] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:21.582695 31662 tablet_replica.cc:330] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: stopping tablet replica
I20260811 09:06:21.583196 31662 raft_consensus.cc:2229] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:21.583781 31662 raft_consensus.cc:2258] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:21.586736 31661 ts_tablet_manager.cc:1905] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:21.590396 31662 ts_tablet_manager.cc:1905] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:21.594600 31661 ts_tablet_manager.cc:1918] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:21.595304 31663 tablet_replica.cc:330] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: stopping tablet replica
I20260811 09:06:21.595857 31663 raft_consensus.cc:2229] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:21.596635 31663 raft_consensus.cc:2258] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:21.597975 31661 log.cc:1196] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/2cf1b991a6394dd0864310a02dca74a0
I20260811 09:06:21.598977 31663 ts_tablet_manager.cc:1905] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:21.602257 31662 ts_tablet_manager.cc:1918] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:21.602494 31662 log.cc:1196] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/2cf1b991a6394dd0864310a02dca74a0
I20260811 09:06:21.603173 31662 ts_tablet_manager.cc:1939] T 2cf1b991a6394dd0864310a02dca74a0 P fa580d02c214482d80160ddbb37536b0: Deleting consensus metadata
I20260811 09:06:21.604210 31661 ts_tablet_manager.cc:1939] T 2cf1b991a6394dd0864310a02dca74a0 P 9d3230fb3801425abe19cc437b195811: Deleting consensus metadata
I20260811 09:06:21.605191 31047 catalog_manager.cc:4841] TS fa580d02c214482d80160ddbb37536b0 (127.9.40.66:46867): tablet 2cf1b991a6394dd0864310a02dca74a0 (table test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]) successfully deleted
I20260811 09:06:21.606210 31663 ts_tablet_manager.cc:1918] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:21.606453 31663 log.cc:1196] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/2cf1b991a6394dd0864310a02dca74a0
I20260811 09:06:21.607065 31663 ts_tablet_manager.cc:1939] T 2cf1b991a6394dd0864310a02dca74a0 P e927a202ebbb4ae4bbd4d9bae3d7e18a: Deleting consensus metadata
I20260811 09:06:21.607568 31048 catalog_manager.cc:4841] TS 9d3230fb3801425abe19cc437b195811 (127.9.40.65:38099): tablet 2cf1b991a6394dd0864310a02dca74a0 (table test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]) successfully deleted
I20260811 09:06:21.608909 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.582894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.9.40.65:38099)
0811 09:06:21.583087 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.583093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.597629 (+ 14536us) server_negotiation.cc:407] Connection header received
0811 09:06:21.597893 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.597899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.598026 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.598133 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:21.598657 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.599861 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.601900 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.602618 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.602718 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.604763 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:21.604775 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:21.604794 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:21.605621 (+ 827us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:21.605681 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:21.605706 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:21.605830 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.608011 (+ 2181us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.608236 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:06:21.608960 31047 catalog_manager.cc:4841] TS e927a202ebbb4ae4bbd4d9bae3d7e18a (127.9.40.67:44837): tablet 2cf1b991a6394dd0864310a02dca74a0 (table test-table [id=3bbabf9bd0c1407a8a00ece3848619b3]) successfully deleted
I20260811 09:06:21.610212 31276 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:21.712347 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31137
I20260811 09:06:21.737092 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31292
I20260811 09:06:21.744143 31622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.737970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.9.40.66:46867)
0811 09:06:21.738140 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.738144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.738163 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:06:21.738313 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.738317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.738406 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.738490 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:21.738908 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.739685 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.741155 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.741550 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.741652 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.742712 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:21.742720 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:21.742735 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:21.743324 (+ 589us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:21.743369 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:21.743393 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:21.743479 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.743661 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.743753 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:06:21.744947 31447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:21.837863 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31292
I20260811 09:06:21.861070 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31456
I20260811 09:06:21.868034 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:21.862030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50044 (local address 127.9.40.67:44837)
0811 09:06:21.862219 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:21.862224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:21.862242 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:06:21.862322 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:21.862326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:21.862418 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:21.862496 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:21.862929 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.863717 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.865145 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:21.865528 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:21.865620 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:21.866662 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:21.866669 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:21.866685 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:21.867289 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:21.867335 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:21.867354 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:21.867437 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:21.867606 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 09:06:21.867696 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:06:21.868786 31603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:21.959367 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31456
I20260811 09:06:21.981145 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31028
I20260811 09:06:21.982249 31099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:22.095920 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31028
2026-08-11T09:06:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2899 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:22 dist-test-slave-cbsf krb5kdc[31671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:22 dist-test-slave-cbsf krb5kdc[31671](info): set up 2 sockets
Aug 11 09:06:22 dist-test-slave-cbsf krb5kdc[31671](info): commencing operation
krb5kdc: starting...
W20260811 09:06:23.332149 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.006s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:23 dist-test-slave-cbsf krb5kdc[31671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:06:23.683877 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44705
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:24.045462 31687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:24.045867 31687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:24.046025 31687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:24.053651 31687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:24.053834 31687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:24.053946 31687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:24.054047 31687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:24.054126 31687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:24.064607 31687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:24.066159 31687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:24.067549 31687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:24.080149 31692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:24.080116 31695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:24.080205 31693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:24.081710 31687 server_base.cc:1034] running on GCE node
I20260811 09:06:24.082556 31687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:24.084081 31687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:24.085314 31687 hybrid_clock.cc:648] HybridClock initialized: now 1786439184085262 us; error 40 us; skew 500 ppm
I20260811 09:06:24.085711 31687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:24 dist-test-slave-cbsf krb5kdc[31671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:24.088932 31687 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:06:24.090544 31687 webserver.cc:460] Webserver started at http://127.9.40.126:40243/ using document root <none> and password file <none>
I20260811 09:06:24.091261 31687 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:24.091408 31687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:24.091725 31687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:24.094899 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "01ade16e485a4e689a91aed6b8a0a944"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "8a5f0df60eaefbb0324b83f50f653d02"
server_key_iv: "6e8b18289474e8046d0af09b24a7190e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.095703 31687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "01ade16e485a4e689a91aed6b8a0a944"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "8a5f0df60eaefbb0324b83f50f653d02"
server_key_iv: "6e8b18289474e8046d0af09b24a7190e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.101249 31687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:06:24.104794 31702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:24.106091 31687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:24.106289 31687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "01ade16e485a4e689a91aed6b8a0a944"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "8a5f0df60eaefbb0324b83f50f653d02"
server_key_iv: "6e8b18289474e8046d0af09b24a7190e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.106494 31687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:24.123435 31687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:24.128290 31687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:24.128616 31687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:24.149253 31687 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44705
I20260811 09:06:24.149290 31763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44705 every 8 connection(s)
I20260811 09:06:24.150782 31687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:24.154816 31764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:24.158651 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31687
I20260811 09:06:24.158854 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:24.159420 9377 external_mini_cluster.cc:1428] Setting key a07527dc2484d19a1861a9df254f1728
I20260811 09:06:24.167732 31764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: Bootstrap starting.
I20260811 09:06:24.171041 31764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:24.172117 31764 log.cc:824] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:06:24 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:24.175681 31764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: No bootstrap required, opened a new log
I20260811 09:06:24.183506 31767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:24.161614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.9.40.126:44705)
0811 09:06:24.162401 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:24.162414 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:24.162455 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:06:24.164241 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:24.164264 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:24.164740 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:24.165175 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:24.166712 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.168104 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.169472 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.170163 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.170363 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:24.175389 (+ 5026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:24.175397 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:24.175412 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:24.175468 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:24.179096 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:24.180002 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:24.180009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:24.180016 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:24.180204 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:24.180649 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:24.180656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:24.180658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:24.181288 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:24.181544 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:24.182022 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 09:06:24.182401 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":326,"threads_started":1}
I20260811 09:06:24.185050 31764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } }
I20260811 09:06:24.185447 31764 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:24.185575 31764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ade16e485a4e689a91aed6b8a0a944, State: Initialized, Role: FOLLOWER
I20260811 09:06:24.186172 31764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } }
I20260811 09:06:24.186429 31764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:24.186581 31764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:24.186772 31764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:24.189103 31764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } }
I20260811 09:06:24.189594 31764 leader_election.cc:302] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01ade16e485a4e689a91aed6b8a0a944; no voters:
I20260811 09:06:24.190244 31764 leader_election.cc:288] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:24.190404 31770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:24.191731 31770 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [term 1 LEADER]: Becoming Leader. State: Replica: 01ade16e485a4e689a91aed6b8a0a944, State: Running, Role: LEADER
I20260811 09:06:24.192425 31770 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } }
I20260811 09:06:24.192582 31764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:24.195019 31772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01ade16e485a4e689a91aed6b8a0a944. Latest consensus state: current_term: 1 leader_uuid: "01ade16e485a4e689a91aed6b8a0a944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } } }
I20260811 09:06:24.195343 31772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:24.195950 31771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01ade16e485a4e689a91aed6b8a0a944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01ade16e485a4e689a91aed6b8a0a944" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44705 } } }
I20260811 09:06:24.196233 31771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:24.202096 31778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:24.207422 31778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:24.217163 31778 catalog_manager.cc:1347] Generated new cluster ID: 55a3629ab5f94223b201f940adae5965
I20260811 09:06:24.217306 31778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:24.238224 31778 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:24.239589 31778 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:24.256784 31778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: Generated new TSK 0
I20260811 09:06:24.257777 31778 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260811 09:06:24.335801 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:24.701092 31794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:24.701400 31794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:24.701501 31794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:24.709403 31794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:24.709527 31794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:24.709695 31794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:24.719951 31794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:24.721401 31794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:24.722754 31794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:24.737223 31801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:24.737337 31800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:24.737314 31803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:24.738332 31794 server_base.cc:1034] running on GCE node
I20260811 09:06:24.739032 31794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:24.740171 31794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:24.741423 31794 hybrid_clock.cc:648] HybridClock initialized: now 1786439184741386 us; error 57 us; skew 500 ppm
I20260811 09:06:24.741780 31794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:24 dist-test-slave-cbsf krb5kdc[31671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:24.745309 31794 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260811 09:06:24.746819 31794 webserver.cc:460] Webserver started at http://127.9.40.65:35679/ using document root <none> and password file <none>
I20260811 09:06:24.747550 31794 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:24.747658 31794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:24.747934 31794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:24.751428 31794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "ba0300ecac474dc58d5bc5b764d2e4b2"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "fe5d85c61413ffb03db3b874144d73b0"
server_key_iv: "688a082eacf2960bd529386f35dfb55e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.752316 31794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "ba0300ecac474dc58d5bc5b764d2e4b2"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "fe5d85c61413ffb03db3b874144d73b0"
server_key_iv: "688a082eacf2960bd529386f35dfb55e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.758092 31794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:06:24.761772 31812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:24.763280 31794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:24.763433 31794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "ba0300ecac474dc58d5bc5b764d2e4b2"
format_stamp: "Formatted at 2026-08-11 09:06:24 on dist-test-slave-cbsf"
server_key: "fe5d85c61413ffb03db3b874144d73b0"
server_key_iv: "688a082eacf2960bd529386f35dfb55e"
server_key_version: "encryptionkey@0"
I20260811 09:06:24.763629 31794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:24.782240 31794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:24.786378 31794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:24.786652 31794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:24.787603 31794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:24.789093 31794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:24.789172 31794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:24.789275 31794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:24.789320 31794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:24.824647 31794 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:40075
I20260811 09:06:24.824744 31946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:40075 every 8 connection(s)
I20260811 09:06:24.826107 31794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:24.831487 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31794
I20260811 09:06:24.831693 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:24.832147 9377 external_mini_cluster.cc:1428] Setting key d477afec3e39d59a1799925e3e67599a
Aug 11 09:06:24 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439184, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:24.851485 31949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:24.828919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:49855 (local address 127.9.40.126:44705)
0811 09:06:24.829396 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:24.829401 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:24.830448 (+ 1047us) server_negotiation.cc:407] Connection header received
0811 09:06:24.832563 (+ 2115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:24.832569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:24.832670 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:24.832821 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:24.834884 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.835719 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.838417 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.839114 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.839253 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:24.844201 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:24.844209 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:24.844216 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:24.844262 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:24.847694 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:24.848772 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:24.848780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:24.848786 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:24.848975 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:24.849613 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:24.849620 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:24.849625 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:24.850065 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:24.850250 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:24.851006 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 09:06:24.851219 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":239,"threads_started":1}
I20260811 09:06:24.852520 31950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:24.829527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44705 (local address 127.9.40.65:49855)
0811 09:06:24.830260 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:24.830309 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:24.832266 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:24.833136 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:24.833149 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:24.833776 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:24.834648 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:24.834678 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.835900 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.835908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:24.838235 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:24.838256 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:24.839377 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:24.839387 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:24.839753 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:24.840607 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:24.840650 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:24.843889 (+ 3239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:24.847886 (+ 3997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:24.847898 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:24.847922 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:24.848565 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:24.849135 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:24.849140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:24.849143 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:24.849438 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:24.850243 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:24.850250 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:24.850621 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 09:06:24.850873 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 09:06:24.851317 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":296,"threads_started":1}
I20260811 09:06:24.854367 31947 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44705
I20260811 09:06:24.854728 31947 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:24.855691 31947 heartbeater.cc:511] Master 127.9.40.126:44705 requested a full tablet report, sending...
I20260811 09:06:24.858220 31720 ts_manager.cc:194] Registered new tserver with Master: ba0300ecac474dc58d5bc5b764d2e4b2 (127.9.40.65:40075)
I20260811 09:06:24.859772 31720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:49855
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260811 09:06:24.912177 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:25.274816 31954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:25.275170 31954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:25.275333 31954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:25.282958 31954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:25.283128 31954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:25.283306 31954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:25.293545 31954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:25.294958 31954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:25.296257 31954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:25.312872 31963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:25.312955 31966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:25.312976 31962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:25.314090 31954 server_base.cc:1034] running on GCE node
I20260811 09:06:25.314767 31954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:25.315769 31954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:25.316987 31954 hybrid_clock.cc:648] HybridClock initialized: now 1786439185316939 us; error 44 us; skew 500 ppm
I20260811 09:06:25.317322 31954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:25 dist-test-slave-cbsf krb5kdc[31671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:25.320653 31954 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260811 09:06:25.322355 31954 webserver.cc:460] Webserver started at http://127.9.40.66:39023/ using document root <none> and password file <none>
I20260811 09:06:25.323092 31954 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:25.323261 31954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:25.323597 31954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:25.326761 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "14794cdd3e6946b1853f658ac83ec263"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4f8ce39a178c545351c66942bbbae234"
server_key_iv: "3d23b8a4c07b11daabf9794c0dcc3a8b"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.327579 31954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "14794cdd3e6946b1853f658ac83ec263"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4f8ce39a178c545351c66942bbbae234"
server_key_iv: "3d23b8a4c07b11daabf9794c0dcc3a8b"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.333292 31954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:06:25.337114 31973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.338526 31954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:25.338733 31954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "14794cdd3e6946b1853f658ac83ec263"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4f8ce39a178c545351c66942bbbae234"
server_key_iv: "3d23b8a4c07b11daabf9794c0dcc3a8b"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.338964 31954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:25.364087 31954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:25.368453 31954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:25.368827 31954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:25.369915 31954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:25.371317 31954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:25.371428 31954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.371573 31954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:25.371647 31954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.406930 31954 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:36751
I20260811 09:06:25.406975 32110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:36751 every 8 connection(s)
I20260811 09:06:25.408270 31954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:25.417198 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 31954
I20260811 09:06:25.417405 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:25.417882 9377 external_mini_cluster.cc:1428] Setting key 65a6c9b03da67e797bec43689190c81e
Aug 11 09:06:25 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:25.429273 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:25.411077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59759 (local address 127.9.40.126:44705)
0811 09:06:25.411527 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:25.411532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:25.412493 (+ 961us) server_negotiation.cc:407] Connection header received
0811 09:06:25.414431 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:25.414440 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:25.414533 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:25.414651 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:25.416243 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.417210 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.418909 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.419407 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.419548 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:25.423162 (+ 3614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:25.423167 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:25.423170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:25.423206 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:25.425990 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:25.426768 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:25.426777 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:25.426781 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:25.426924 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:25.427369 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:25.427374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:25.427377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:25.427676 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:25.427836 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:25.428580 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 09:06:25.428858 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":209,"threads_started":1}
I20260811 09:06:25.430322 32114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:25.411613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44705 (local address 127.9.40.66:59759)
0811 09:06:25.412328 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:25.412364 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:25.414171 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:25.414881 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:25.414889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:25.415399 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:25.416071 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:25.416090 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.417398 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.417407 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:25.418754 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:25.418767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.419652 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.419660 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:25.419891 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:25.420593 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:25.420627 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:25.422898 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:25.426145 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:25.426156 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:25.426176 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:25.426637 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:25.427040 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:25.427045 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:25.427049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:25.427248 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:25.427810 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:25.427819 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:25.428193 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 09:06:25.428481 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 09:06:25.429112 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":279,"threads_started":1}
I20260811 09:06:25.431879 32111 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44705
I20260811 09:06:25.432251 32111 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:25.433187 32111 heartbeater.cc:511] Master 127.9.40.126:44705 requested a full tablet report, sending...
I20260811 09:06:25.434830 31720 ts_manager.cc:194] Registered new tserver with Master: 14794cdd3e6946b1853f658ac83ec263 (127.9.40.66:36751)
I20260811 09:06:25.435650 31720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:59759
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260811 09:06:25.492147 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:25.847977 32118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:25.848354 32118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:25.848536 32118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:25.856210 32118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:25.856374 32118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:25.856564 32118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:25.863915 31947 heartbeater.cc:503] Master 127.9.40.126:44705 was elected leader, sending a full tablet report...
I20260811 09:06:25.867219 32118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:25.868615 32118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:25.870033 32118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:25.883533 32123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:25.883574 32127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:25.883641 32125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:25.884851 32118 server_base.cc:1034] running on GCE node
I20260811 09:06:25.885517 32118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:25.886488 32118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:25.887732 32118 hybrid_clock.cc:648] HybridClock initialized: now 1786439185887689 us; error 55 us; skew 500 ppm
I20260811 09:06:25.888070 32118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:25 dist-test-slave-cbsf krb5kdc[31671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:25.891399 32118 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260811 09:06:25.892982 32118 webserver.cc:460] Webserver started at http://127.9.40.67:33971/ using document root <none> and password file <none>
I20260811 09:06:25.893746 32118 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:25.893888 32118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:25.894202 32118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:25.897245 32118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "0cb9b5a021fb48889cb208d14e65168c"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4a32f4aa0c6078f68895b7ede6355c27"
server_key_iv: "0abe53ce7edcbfac0d29e8b76655938f"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.898053 32118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "0cb9b5a021fb48889cb208d14e65168c"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4a32f4aa0c6078f68895b7ede6355c27"
server_key_iv: "0abe53ce7edcbfac0d29e8b76655938f"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.903553 32118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 09:06:25.907014 32137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.908402 32118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:06:25.908595 32118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "0cb9b5a021fb48889cb208d14e65168c"
format_stamp: "Formatted at 2026-08-11 09:06:25 on dist-test-slave-cbsf"
server_key: "4a32f4aa0c6078f68895b7ede6355c27"
server_key_iv: "0abe53ce7edcbfac0d29e8b76655938f"
server_key_version: "encryptionkey@0"
I20260811 09:06:25.908856 32118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:25.921757 32118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:25.926331 32118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:25.926651 32118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:25.927676 32118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:25.929136 32118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:25.929211 32118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.929374 32118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:25.929425 32118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:25.964095 32118 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:41679
I20260811 09:06:25.964143 32273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:41679 every 8 connection(s)
I20260811 09:06:25.965420 32118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:25.966667 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 32118
I20260811 09:06:25.966868 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:25.967451 9377 external_mini_cluster.cc:1428] Setting key 6018de80264a52dca2bf9dc7cc1f760d
Aug 11 09:06:25 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439185, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:25.986526 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:25.968556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:50979 (local address 127.9.40.126:44705)
0811 09:06:25.969197 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:25.969202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:25.970334 (+ 1132us) server_negotiation.cc:407] Connection header received
0811 09:06:25.972294 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:25.972298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:25.972364 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:25.972453 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:25.974008 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.974778 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.976292 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.976667 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.976807 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:25.980710 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:25.980717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:25.980721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:25.980759 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:25.983860 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:25.984536 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:25.984539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:25.984541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:25.984654 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:25.985064 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:25.985067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:25.985069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:25.985330 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:25.985445 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:25.986097 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 09:06:25.986252 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":340,"threads_started":1}
I20260811 09:06:25.987480 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:25.969262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44705 (local address 127.9.40.67:50979)
0811 09:06:25.970174 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:25.970230 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:25.972045 (+ 1815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:25.972660 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:25.972669 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:25.973289 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:25.973857 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:25.973875 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.974910 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.974914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:25.976176 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:25.976189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:25.976918 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:25.976924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:25.977143 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:25.977843 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:06:25.977866 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:25.980421 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:25.983988 (+ 3567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:25.983995 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:25.984007 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:25.984440 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:25.984792 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:06:25.984795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:06:25.984798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:06:25.984974 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:06:25.985451 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:25.985457 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:25.985762 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 09:06:25.986001 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 09:06:25.986398 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":396,"threads_started":1}
I20260811 09:06:25.988879 32274 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44705
I20260811 09:06:25.989190 32274 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:25.989890 32274 heartbeater.cc:511] Master 127.9.40.126:44705 requested a full tablet report, sending...
I20260811 09:06:25.991343 31720 ts_manager.cc:194] Registered new tserver with Master: 0cb9b5a021fb48889cb208d14e65168c (127.9.40.67:41679)
I20260811 09:06:25.992168 31720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:50979
I20260811 09:06:25.995267 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:26.012033 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:25.999279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.9.40.126:44705)
0811 09:06:25.999484 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:25.999491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:25.999875 (+ 384us) server_negotiation.cc:407] Connection header received
0811 09:06:26.000028 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.000032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.000097 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.000168 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:26.001739 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.002465 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.003876 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.004277 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.004370 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.006782 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:26.006787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:26.006790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:26.006820 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:26.009356 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.010115 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.010119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.010121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.010264 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.010636 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.010639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.010641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.010947 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:26.011121 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.011520 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.011704 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:06:26.016319 31720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:26.023370 31720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:26.046433 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.036484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.9.40.65:40075)
0811 09:06:26.037257 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.037267 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.037347 (+ 80us) server_negotiation.cc:407] Connection header received
0811 09:06:26.037631 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.037639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.037896 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.038104 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:26.039156 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.040856 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.043829 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.045220 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.045370 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.045632 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.045898 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.046080 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":410,"threads_started":1}
I20260811 09:06:26.046451 32287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.036370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40075 (local address 127.0.0.1:45538)
0811 09:06:26.037185 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:26.037227 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:26.037432 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:26.038456 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:26.038475 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:26.038512 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:26.038957 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.038975 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.041035 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.041042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.043623 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.043637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.045435 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.045451 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.045637 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.045686 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 09:06:26.045835 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 09:06:26.046062 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":2,"spinlock_wait_cycles":24320,"thread_start_us":770,"threads_started":1}
I20260811 09:06:26.049343 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.037926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:41679 (local address 127.0.0.1:45036)
0811 09:06:26.038514 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:26.038535 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:26.038716 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:26.041007 (+ 2291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:26.041012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:26.041039 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:26.041583 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.041602 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.043744 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.043750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.045876 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.045889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.048418 (+ 2529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.048424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.048613 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.048657 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 09:06:26.048784 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 09:06:26.048951 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":81,"spinlock_wait_cycles":1280,"thread_start_us":292,"threads_started":1}
I20260811 09:06:26.050249 32290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.038312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.9.40.67:41679)
0811 09:06:26.040316 (+ 2004us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.040328 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.040369 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:06:26.040503 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.040515 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.040815 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.041007 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:26.041853 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.043590 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.046713 (+ 3123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.048227 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.048370 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.048584 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.049732 (+ 1148us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.049909 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1776,"thread_start_us":2083,"threads_started":1}
I20260811 09:06:26.050907 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.038546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:36751 (local address 127.0.0.1:53430)
0811 09:06:26.039503 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:26.039526 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:26.039660 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:26.041622 (+ 1962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:26.041629 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:26.041664 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:26.042210 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.042222 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.044151 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.044156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.046456 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.046474 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.050028 (+ 3554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.050035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.050205 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.050245 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:06:26.050360 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 09:06:26.050514 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"mutex_wait_us":120,"thread_start_us":760,"threads_started":1}
I20260811 09:06:26.050866 31868 tablet_service.cc:1467] Processing CreateTablet for tablet 6b510fd3f7ca443d8010850695ce924d (DEFAULT_TABLE table=test-table [id=81b7ce90143a4e1a8b019b943bb8e794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:26.051955 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.039115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.9.40.66:36751)
0811 09:06:26.041038 (+ 1923us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.041045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.041071 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:06:26.041184 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.041195 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.041428 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.041618 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:26.042391 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.043987 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.048671 (+ 4684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.049828 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.049977 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.050201 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.051500 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.051691 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1745,"thread_start_us":1616,"threads_started":1}
I20260811 09:06:26.052556 31868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b510fd3f7ca443d8010850695ce924d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:26.054275 32193 tablet_service.cc:1467] Processing CreateTablet for tablet 6b510fd3f7ca443d8010850695ce924d (DEFAULT_TABLE table=test-table [id=81b7ce90143a4e1a8b019b943bb8e794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:26.055334 32034 tablet_service.cc:1467] Processing CreateTablet for tablet 6b510fd3f7ca443d8010850695ce924d (DEFAULT_TABLE table=test-table [id=81b7ce90143a4e1a8b019b943bb8e794]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:26.055797 32193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b510fd3f7ca443d8010850695ce924d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:26.056416 32034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b510fd3f7ca443d8010850695ce924d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:26.068465 32295 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Bootstrap starting.
I20260811 09:06:26.068980 32294 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Bootstrap starting.
I20260811 09:06:26.070421 32296 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Bootstrap starting.
I20260811 09:06:26.072124 32294 tablet_bootstrap.cc:654] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:26.073172 32296 tablet_bootstrap.cc:654] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:26.073490 32294 log.cc:824] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:26.074230 32296 log.cc:824] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:26.075711 32295 tablet_bootstrap.cc:654] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:26.076987 32295 log.cc:824] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:26.077044 32294 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: No bootstrap required, opened a new log
I20260811 09:06:26.077363 32294 ts_tablet_manager.cc:1397] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:06:26.079370 32296 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: No bootstrap required, opened a new log
I20260811 09:06:26.079715 32296 ts_tablet_manager.cc:1397] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:06:26.083420 32295 tablet_bootstrap.cc:492] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: No bootstrap required, opened a new log
I20260811 09:06:26.083702 32295 ts_tablet_manager.cc:1397] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.003s
I20260811 09:06:26.085341 32294 raft_consensus.cc:348] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.085721 32294 raft_consensus.cc:374] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:26.085827 32294 raft_consensus.cc:729] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14794cdd3e6946b1853f658ac83ec263, State: Initialized, Role: FOLLOWER
I20260811 09:06:26.086493 32294 consensus_queue.cc:260] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.089468 32111 heartbeater.cc:503] Master 127.9.40.126:44705 was elected leader, sending a full tablet report...
I20260811 09:06:26.090221 32294 ts_tablet_manager.cc:1428] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260811 09:06:26.090750 32296 raft_consensus.cc:348] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.091200 32296 raft_consensus.cc:374] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:26.091305 32296 raft_consensus.cc:729] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cb9b5a021fb48889cb208d14e65168c, State: Initialized, Role: FOLLOWER
I20260811 09:06:26.091915 32296 consensus_queue.cc:260] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.095410 32295 raft_consensus.cc:348] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.095975 32295 raft_consensus.cc:374] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:26.096099 32295 raft_consensus.cc:729] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba0300ecac474dc58d5bc5b764d2e4b2, State: Initialized, Role: FOLLOWER
I20260811 09:06:26.096895 32295 consensus_queue.cc:260] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.097434 32274 heartbeater.cc:503] Master 127.9.40.126:44705 was elected leader, sending a full tablet report...
I20260811 09:06:26.098210 32296 ts_tablet_manager.cc:1428] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Time spent starting tablet: real 0.018s user 0.011s sys 0.008s
I20260811 09:06:26.098658 32295 ts_tablet_manager.cc:1428] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260811 09:06:26.160439 32112 tablet.cc:2369] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:26.173249 32302 raft_consensus.cc:482] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:26.173605 32302 raft_consensus.cc:504] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.175379 32302 leader_election.cc:288] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba0300ecac474dc58d5bc5b764d2e4b2 (127.9.40.65:40075), 0cb9b5a021fb48889cb208d14e65168c (127.9.40.67:41679)
I20260811 09:06:26.182547 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.175943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:38069 (local address 127.9.40.65:40075)
0811 09:06:26.176204 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.176211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.176396 (+ 185us) server_negotiation.cc:407] Connection header received
0811 09:06:26.176622 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.176631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.176763 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.176865 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:26.177716 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.178657 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.180603 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.181564 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.181668 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.181744 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.182152 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.182301 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:06:26.182588 32305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.175634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40075 (local address 127.9.40.66:38069)
0811 09:06:26.176276 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:26.176294 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:26.176480 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:26.176933 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:26.176939 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:26.176980 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:26.177490 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.177511 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.178782 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.178787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.180437 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.180449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.181780 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.181786 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.181967 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.182016 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 09:06:26.182124 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 09:06:26.182273 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":310,"threads_started":1}
I20260811 09:06:26.183030 32306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.176429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:41679 (local address 127.9.40.66:39845)
0811 09:06:26.177056 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:26.177069 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:26.177197 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:26.177611 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:26.177617 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:26.177654 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:26.178096 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.178110 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.179305 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.179313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.180891 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:26.180909 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.182191 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.182197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:26.182395 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.182428 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:06:26.182567 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 09:06:26.182726 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":471,"threads_started":1}
I20260811 09:06:26.183028 32290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.176681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39845 (local address 127.9.40.67:41679)
0811 09:06:26.176905 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.176911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.177165 (+ 254us) server_negotiation.cc:407] Connection header received
0811 09:06:26.177330 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.177338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.177434 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.177540 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:26.178259 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.179168 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.181046 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.181944 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.182071 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.182180 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.182618 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.182751 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 09:06:26.183975 31891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b510fd3f7ca443d8010850695ce924d" candidate_uuid: "14794cdd3e6946b1853f658ac83ec263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" is_pre_election: true
I20260811 09:06:26.183992 32216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b510fd3f7ca443d8010850695ce924d" candidate_uuid: "14794cdd3e6946b1853f658ac83ec263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb9b5a021fb48889cb208d14e65168c" is_pre_election: true
I20260811 09:06:26.184602 31891 raft_consensus.cc:2454] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14794cdd3e6946b1853f658ac83ec263 in term 0.
I20260811 09:06:26.184602 32216 raft_consensus.cc:2454] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14794cdd3e6946b1853f658ac83ec263 in term 0.
I20260811 09:06:26.185333 31977 leader_election.cc:302] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0cb9b5a021fb48889cb208d14e65168c, 14794cdd3e6946b1853f658ac83ec263; no voters:
I20260811 09:06:26.185674 32302 raft_consensus.cc:2781] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:26.185827 32302 raft_consensus.cc:482] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:26.185926 32302 raft_consensus.cc:3037] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:26.188086 32302 raft_consensus.cc:504] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.188776 32302 leader_election.cc:288] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [CANDIDATE]: Term 1 election: Requested vote from peers ba0300ecac474dc58d5bc5b764d2e4b2 (127.9.40.65:40075), 0cb9b5a021fb48889cb208d14e65168c (127.9.40.67:41679)
I20260811 09:06:26.189273 31891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b510fd3f7ca443d8010850695ce924d" candidate_uuid: "14794cdd3e6946b1853f658ac83ec263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2"
I20260811 09:06:26.189312 32216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b510fd3f7ca443d8010850695ce924d" candidate_uuid: "14794cdd3e6946b1853f658ac83ec263" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cb9b5a021fb48889cb208d14e65168c"
I20260811 09:06:26.189514 31891 raft_consensus.cc:3037] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:26.189509 32216 raft_consensus.cc:3037] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:26.192802 32216 raft_consensus.cc:2454] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14794cdd3e6946b1853f658ac83ec263 in term 1.
I20260811 09:06:26.192803 31891 raft_consensus.cc:2454] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14794cdd3e6946b1853f658ac83ec263 in term 1.
I20260811 09:06:26.193234 31977 leader_election.cc:302] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14794cdd3e6946b1853f658ac83ec263, ba0300ecac474dc58d5bc5b764d2e4b2; no voters:
I20260811 09:06:26.193557 32302 raft_consensus.cc:2781] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:26.193990 32302 raft_consensus.cc:686] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 1 LEADER]: Becoming Leader. State: Replica: 14794cdd3e6946b1853f658ac83ec263, State: Running, Role: LEADER
I20260811 09:06:26.194567 32302 consensus_queue.cc:237] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } }
I20260811 09:06:26.199152 31719 catalog_manager.cc:5495] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14794cdd3e6946b1853f658ac83ec263 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "14794cdd3e6946b1853f658ac83ec263" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14794cdd3e6946b1853f658ac83ec263" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 36751 } health_report { overall_health: HEALTHY } } }
W20260811 09:06:26.217041 32275 tablet.cc:2369] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:26.239624 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.234502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.9.40.66:36751)
0811 09:06:26.234706 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.234711 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.234731 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:06:26.234825 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.234829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.234918 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.235043 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:26.235395 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.236104 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.237505 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.237906 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.237996 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.238443 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:26.239003 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:26.239112 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.239214 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.239285 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:06:26.248159 31891 raft_consensus.cc:1261] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 1 FOLLOWER]: Refusing update from remote peer 14794cdd3e6946b1853f658ac83ec263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:26.248153 32216 raft_consensus.cc:1261] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 1 FOLLOWER]: Refusing update from remote peer 14794cdd3e6946b1853f658ac83ec263: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:26.249164 32307 consensus_queue.cc:1035] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cb9b5a021fb48889cb208d14e65168c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 41679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:26.249538 32302 consensus_queue.cc:1035] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba0300ecac474dc58d5bc5b764d2e4b2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:26.260845 32314 mvcc.cc:205] Tried to move back new op lower bound from 7317254906862338048 to 7317254906656821248. Current Snapshot: MvccSnapshot[applied={T|T < 7317254906862338048}]
I20260811 09:06:26.275771 31720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:26.276070 31720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:26.279627 31720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01ade16e485a4e689a91aed6b8a0a944: Sending DeleteTablet for 3 replicas of tablet 6b510fd3f7ca443d8010850695ce924d
I20260811 09:06:26.281006 32193 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b510fd3f7ca443d8010850695ce924d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:45036
I20260811 09:06:26.281008 31868 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b510fd3f7ca443d8010850695ce924d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:45538
I20260811 09:06:26.281141 32034 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b510fd3f7ca443d8010850695ce924d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:26 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53430
I20260811 09:06:26.282955 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31794
I20260811 09:06:26.295198 32327 tablet_replica.cc:330] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: stopping tablet replica
I20260811 09:06:26.295390 32328 tablet_replica.cc:330] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: stopping tablet replica
I20260811 09:06:26.295724 32327 raft_consensus.cc:2229] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:26.295851 32328 raft_consensus.cc:2229] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:26.296321 32327 raft_consensus.cc:2258] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:26.296567 32328 raft_consensus.cc:2258] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:26.298748 32327 ts_tablet_manager.cc:1905] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:26.298761 32328 ts_tablet_manager.cc:1905] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:26.300480 32326 tablet_replica.cc:330] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: stopping tablet replica
I20260811 09:06:26.301043 32326 raft_consensus.cc:2229] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:26.301582 32326 raft_consensus.cc:2258] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:26.306108 32326 ts_tablet_manager.cc:1905] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:26.306183 32327 ts_tablet_manager.cc:1918] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:26.306373 32327 log.cc:1196] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/6b510fd3f7ca443d8010850695ce924d
I20260811 09:06:26.306968 32327 ts_tablet_manager.cc:1939] T 6b510fd3f7ca443d8010850695ce924d P 0cb9b5a021fb48889cb208d14e65168c: Deleting consensus metadata
I20260811 09:06:26.307053 32328 ts_tablet_manager.cc:1918] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:26.307225 32328 log.cc:1196] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/6b510fd3f7ca443d8010850695ce924d
I20260811 09:06:26.307811 32328 ts_tablet_manager.cc:1939] T 6b510fd3f7ca443d8010850695ce924d P 14794cdd3e6946b1853f658ac83ec263: Deleting consensus metadata
I20260811 09:06:26.309068 31706 catalog_manager.cc:4841] TS 0cb9b5a021fb48889cb208d14e65168c (127.9.40.67:41679): tablet 6b510fd3f7ca443d8010850695ce924d (table test-table [id=81b7ce90143a4e1a8b019b943bb8e794]) successfully deleted
I20260811 09:06:26.309856 31706 catalog_manager.cc:4841] TS 14794cdd3e6946b1853f658ac83ec263 (127.9.40.66:36751): tablet 6b510fd3f7ca443d8010850695ce924d (table test-table [id=81b7ce90143a4e1a8b019b943bb8e794]) successfully deleted
I20260811 09:06:26.312453 32326 ts_tablet_manager.cc:1918] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:26.312628 32326 log.cc:1196] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/6b510fd3f7ca443d8010850695ce924d
I20260811 09:06:26.313199 32326 ts_tablet_manager.cc:1939] T 6b510fd3f7ca443d8010850695ce924d P ba0300ecac474dc58d5bc5b764d2e4b2: Deleting consensus metadata
Aug 11 09:06:26 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260811 09:06:26.314826 31706 catalog_manager.cc:4841] TS ba0300ecac474dc58d5bc5b764d2e4b2 (127.9.40.65:40075): tablet 6b510fd3f7ca443d8010850695ce924d (table test-table [id=81b7ce90143a4e1a8b019b943bb8e794]) successfully deleted
I20260811 09:06:26.321300 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.284758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.9.40.65:40075)
0811 09:06:26.284996 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.285006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.294294 (+ 9288us) server_negotiation.cc:407] Connection header received
0811 09:06:26.294496 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.294510 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.294679 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.296606 (+ 1927us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:26.296843 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.298238 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.301988 (+ 3750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.302780 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.302906 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.314958 (+ 12052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:26.314973 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:26.314993 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:26.315038 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:26.318062 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.319123 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.319131 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.319133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.319278 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.319731 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.319735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.319736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.320108 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:26.320339 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.320798 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.320977 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:06:26.322010 31939 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:26.419739 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31794
I20260811 09:06:26.452512 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31954
Aug 11 09:06:26 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
I20260811 09:06:26.473069 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.453442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.9.40.66:36751)
0811 09:06:26.453672 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.453680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.453703 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:06:26.453768 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.453774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.453836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.453954 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:26.455277 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.456106 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.457465 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.457939 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.458075 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.466765 (+ 8690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:26.466782 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:26.466800 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:26.466850 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:26.470090 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.470971 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.470978 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.470980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.471141 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.471544 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.471546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.471548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.471887 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:26.472159 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.472481 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.472671 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:06:26.473806 32105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:26.611227 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31954
I20260811 09:06:26.637287 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 32118
Aug 11 09:06:26 dist-test-slave-cbsf krb5kdc[31671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260811 09:06:26.652470 32290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:26.638017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45042 (local address 127.9.40.67:41679)
0811 09:06:26.638276 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:26.638282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:26.638302 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:06:26.638468 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:26.638473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:26.638541 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:26.638648 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:26.640003 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.640809 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.642168 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:26.642569 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:26.642711 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:26.647335 (+ 4624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:26.647354 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:26.647372 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:26.647413 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:26.650090 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.650727 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.650734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.650736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.650879 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:26.651220 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:26.651223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:26.651225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:26.651576 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:26.651798 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:26.652046 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 09:06:26.652211 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260811 09:06:26.653455 32267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:26.745508 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 32118
I20260811 09:06:26.769160 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 31687
I20260811 09:06:26.770304 31759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:26.885560 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 31687
2026-08-11T09:06:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4791 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T09:06:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:26Z Disabled control of system clock
I20260811 09:06:26.990007 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42973
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:42973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:27.353055 32338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:27.353438 32338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:27.353603 32338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:27.361063 32338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:27.361233 32338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:27.361322 32338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:27.361424 32338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:27.361505 32338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:27.371086 32338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:42973
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:27.372507 32338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:27.374432 32338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:27.387154 32344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:27.387172 32347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:27.387223 32345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:27.388093 32338 server_base.cc:1034] running on GCE node
I20260811 09:06:27.388952 32338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:27.390436 32338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:27.391676 32338 hybrid_clock.cc:648] HybridClock initialized: now 1786439187391607 us; error 60 us; skew 500 ppm
I20260811 09:06:27.392032 32338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:27.397553 32338 webserver.cc:460] Webserver started at http://127.9.40.126:37333/ using document root <none> and password file <none>
I20260811 09:06:27.398281 32338 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:27.398413 32338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:27.398741 32338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:27.401813 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "51ba08605dea41e0b662df7ca2e74378"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "ccf2088e83a3c7898b7132a139d3d9f9"
server_key_iv: "ad29537cd798eb165f49a2795bec93bd"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.402634 32338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "51ba08605dea41e0b662df7ca2e74378"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "ccf2088e83a3c7898b7132a139d3d9f9"
server_key_iv: "ad29537cd798eb165f49a2795bec93bd"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.408102 32338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:06:27.411782 32353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:27.413182 32338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:27.413370 32338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "51ba08605dea41e0b662df7ca2e74378"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "ccf2088e83a3c7898b7132a139d3d9f9"
server_key_iv: "ad29537cd798eb165f49a2795bec93bd"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.413610 32338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:27.446154 32338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:27.447150 32338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:27.447455 32338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:27.470827 32338 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:42973
I20260811 09:06:27.470862 32411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:42973 every 8 connection(s)
I20260811 09:06:27.472317 32338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:27.475831 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 32338
I20260811 09:06:27.476039 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:27.476352 32413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:27.476569 9377 external_mini_cluster.cc:1428] Setting key e6d822a4a989eda3a15b188b13f9f3d3
I20260811 09:06:27.492014 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:27.478742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.9.40.126:42973)
0811 09:06:27.479848 (+ 1106us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:27.479863 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:27.479921 (+ 58us) server_negotiation.cc:407] Connection header received
0811 09:06:27.481657 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:27.481687 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:27.482291 (+ 604us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:27.482779 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:27.483335 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:27.485426 (+ 2091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:27.486772 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:27.487845 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:27.488021 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:27.489157 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:27.489169 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:27.489192 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:27.489956 (+ 764us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:27.490066 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:27.490094 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:27.490213 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:27.490367 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:06:27.490702 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":351,"threads_started":1}
I20260811 09:06:27.492332 32413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: Bootstrap starting.
I20260811 09:06:27.495507 32413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:27.496551 32413 log.cc:824] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:27.499306 32413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: No bootstrap required, opened a new log
I20260811 09:06:27.506916 32413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } }
I20260811 09:06:27.507341 32413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:27.507452 32413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ba08605dea41e0b662df7ca2e74378, State: Initialized, Role: FOLLOWER
I20260811 09:06:27.508019 32413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } }
I20260811 09:06:27.508267 32413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:27.508414 32413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:27.508594 32413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:27.510862 32413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } }
I20260811 09:06:27.511387 32413 leader_election.cc:302] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51ba08605dea41e0b662df7ca2e74378; no voters:
I20260811 09:06:27.511983 32413 leader_election.cc:288] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:27.512140 32420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:27.512884 32420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [term 1 LEADER]: Becoming Leader. State: Replica: 51ba08605dea41e0b662df7ca2e74378, State: Running, Role: LEADER
I20260811 09:06:27.513475 32420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } }
I20260811 09:06:27.514138 32413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:27.515981 32421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51ba08605dea41e0b662df7ca2e74378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } } }
I20260811 09:06:27.516405 32421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:27.517146 32422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ba08605dea41e0b662df7ca2e74378. Latest consensus state: current_term: 1 leader_uuid: "51ba08605dea41e0b662df7ca2e74378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ba08605dea41e0b662df7ca2e74378" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42973 } } }
I20260811 09:06:27.517586 32422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:27.519176 32425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:27.524245 32425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:27.533347 32425 catalog_manager.cc:1347] Generated new cluster ID: 66ff013733cf4ff8a7f593df1495386f
I20260811 09:06:27.533488 32425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:27.554577 32425 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:27.555932 32425 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:27.567442 32425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: Generated new TSK 0
I20260811 09:06:27.568327 32425 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:27.575829 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--builtin_ntp_servers=127.9.40.84:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:06:27.932595 32440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:27.933003 32440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:27.933164 32440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:27.940872 32440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:27.941015 32440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:27.941198 32440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260811 09:06:27.950742 32440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:27.952175 32440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:27.954083 32440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:27.967676 32448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:27.967768 32447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:27.967873 32450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:27.968478 32440 server_base.cc:1034] running on GCE node
I20260811 09:06:27.969151 32440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:27.970058 32440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:27.971268 32440 hybrid_clock.cc:648] HybridClock initialized: now 1786439187971226 us; error 33 us; skew 500 ppm
I20260811 09:06:27.971558 32440 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:27.973942 32440 webserver.cc:460] Webserver started at http://127.9.40.65:37185/ using document root <none> and password file <none>
I20260811 09:06:27.974615 32440 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:27.974706 32440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:27.974970 32440 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:27.977988 32440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data/instance:
uuid: "6ffb5af257c043beaf8eca304fb727c7"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "08b1b70c55b3f31a979d7f1118bfcccb"
server_key_iv: "611b4f48b47bf6880d0745429fd3fc80"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.978755 32440 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "6ffb5af257c043beaf8eca304fb727c7"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "08b1b70c55b3f31a979d7f1118bfcccb"
server_key_iv: "611b4f48b47bf6880d0745429fd3fc80"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.983947 32440 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:06:27.987488 32457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:27.988850 32440 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:06:27.989022 32440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal
uuid: "6ffb5af257c043beaf8eca304fb727c7"
format_stamp: "Formatted at 2026-08-11 09:06:27 on dist-test-slave-cbsf"
server_key: "08b1b70c55b3f31a979d7f1118bfcccb"
server_key_iv: "611b4f48b47bf6880d0745429fd3fc80"
server_key_version: "encryptionkey@0"
I20260811 09:06:27.989254 32440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:28.029210 32440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:28.030128 32440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:28.030427 32440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:28.031486 32440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:28.033155 32440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:28.033285 32440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:28.033423 32440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:28.033510 32440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:28.068635 32440 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38617
I20260811 09:06:28.068751 32592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38617 every 8 connection(s)
I20260811 09:06:28.070154 32440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/data/info.pb
I20260811 09:06:28.079936 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 32440
I20260811 09:06:28.080129 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal/instance
I20260811 09:06:28.080772 9377 external_mini_cluster.cc:1428] Setting key 229b9d267f99d930bdb7553b3295e6e1
I20260811 09:06:28.083591 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:28.072878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33555 (local address 127.9.40.126:42973)
0811 09:06:28.073339 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:28.073344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:28.074606 (+ 1262us) server_negotiation.cc:407] Connection header received
0811 09:06:28.076512 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:28.076518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:28.076613 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:28.076753 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:28.077881 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.078627 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.080330 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.081029 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.081189 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:28.082642 (+ 1453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:28.082651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:28.082658 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:28.082684 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:28.082738 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:28.082755 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:28.082841 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:28.083141 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 09:06:28.083245 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":238,"threads_started":1}
I20260811 09:06:28.084106 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--builtin_ntp_servers=127.9.40.84:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:28.084677 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:28.073505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42973 (local address 127.9.40.65:33555)
0811 09:06:28.074442 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:28.074478 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:28.076227 (+ 1749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:28.076991 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:28.077000 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:28.077098 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:28.077705 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:28.077724 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.078805 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.078810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:28.080147 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:28.080167 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.081275 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.081280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:28.081502 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:28.082237 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:28.082241 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:28.082297 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:28.082410 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:28.082867 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:28.082880 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:28.082888 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:06:28.083094 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 09:06:28.083442 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":298,"threads_started":1}
I20260811 09:06:28.086299 32593 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42973
I20260811 09:06:28.086625 32593 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:28.087414 32593 heartbeater.cc:511] Master 127.9.40.126:42973 requested a full tablet report, sending...
I20260811 09:06:28.089545 32373 ts_manager.cc:194] Registered new tserver with Master: 6ffb5af257c043beaf8eca304fb727c7 (127.9.40.65:38617)
I20260811 09:06:28.091035 32373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:33555
W20260811 09:06:28.448612 32597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:28.449029 32597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:28.449141 32597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:28.456620 32597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:28.456820 32597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:28.456954 32597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260811 09:06:28.466941 32597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:28.468413 32597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:28.470558 32597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:28.486445 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:28.486477 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:28.487259 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:28.488073 32597 server_base.cc:1034] running on GCE node
I20260811 09:06:28.488801 32597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:28.489857 32597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:28.491102 32597 hybrid_clock.cc:648] HybridClock initialized: now 1786439188491043 us; error 69 us; skew 500 ppm
I20260811 09:06:28.491452 32597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:28.494087 32597 webserver.cc:460] Webserver started at http://127.9.40.66:40735/ using document root <none> and password file <none>
I20260811 09:06:28.494861 32597 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:28.494992 32597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:28.495337 32597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:28.498867 32597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data/instance:
uuid: "f0023292904345039282b146ffca3c00"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "4f0ef334818a70672bd54960f6e9f493"
server_key_iv: "e822f72459b8ae8a505ea23e9302b634"
server_key_version: "encryptionkey@0"
I20260811 09:06:28.499755 32597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "f0023292904345039282b146ffca3c00"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "4f0ef334818a70672bd54960f6e9f493"
server_key_iv: "e822f72459b8ae8a505ea23e9302b634"
server_key_version: "encryptionkey@0"
I20260811 09:06:28.505440 32597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 09:06:28.509363 32613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:28.510732 32597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:28.510931 32597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal
uuid: "f0023292904345039282b146ffca3c00"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "4f0ef334818a70672bd54960f6e9f493"
server_key_iv: "e822f72459b8ae8a505ea23e9302b634"
server_key_version: "encryptionkey@0"
I20260811 09:06:28.511164 32597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:28.536244 32597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:28.537256 32597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:28.537567 32597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:28.538643 32597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:28.540248 32597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:28.540401 32597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:28.540544 32597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:28.540637 32597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:28.575656 32597 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39667
I20260811 09:06:28.575716 32745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39667 every 8 connection(s)
I20260811 09:06:28.577070 32597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/data/info.pb
I20260811 09:06:28.580664 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 32597
I20260811 09:06:28.580883 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal/instance
I20260811 09:06:28.581352 9377 external_mini_cluster.cc:1428] Setting key 6524d91eaba05a4d01ff634adcc3deb9
I20260811 09:06:28.584232 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--builtin_ntp_servers=127.9.40.84:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:06:28.590932 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:28.580136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:40617 (local address 127.9.40.126:42973)
0811 09:06:28.580372 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:28.580382 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:28.582083 (+ 1701us) server_negotiation.cc:407] Connection header received
0811 09:06:28.584196 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:28.584201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:28.584294 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:28.584411 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:28.585767 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.586514 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.588102 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.588526 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.588659 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:28.590089 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:28.590093 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:28.590096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:28.590116 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:28.590147 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:28.590159 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:28.590255 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:28.590497 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 09:06:28.590600 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:06:28.592012 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:28.580732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42973 (local address 127.9.40.66:40617)
0811 09:06:28.581901 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:28.581948 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:28.583914 (+ 1966us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:28.584666 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:28.584675 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:28.584839 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:28.585597 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:28.585619 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.586677 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.586682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:28.587947 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:28.587959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:28.588790 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:28.588800 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:28.589028 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:28.589761 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:28.589763 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:28.589805 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:28.589889 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:28.590256 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:28.590270 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:28.590280 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:06:28.590470 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 09:06:28.590890 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":391,"threads_started":1}
I20260811 09:06:28.593540 32746 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42973
I20260811 09:06:28.593928 32746 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:28.594893 32746 heartbeater.cc:511] Master 127.9.40.126:42973 requested a full tablet report, sending...
I20260811 09:06:28.596578 32373 ts_manager.cc:194] Registered new tserver with Master: f0023292904345039282b146ffca3c00 (127.9.40.66:39667)
I20260811 09:06:28.597837 32373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:40617
W20260811 09:06:28.947486 32750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:28.947875 32750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:28.948038 32750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:28.955621 32750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:28.955801 32750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:28.955976 32750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260811 09:06:28.965581 32750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:42973
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:28.967038 32750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:28.968948 32750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:28.983078 32756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:28.983229 32757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:28.983256 32760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:28.984097 32750 server_base.cc:1034] running on GCE node
I20260811 09:06:28.984862 32750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:28.985903 32750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:28.987159 32750 hybrid_clock.cc:648] HybridClock initialized: now 1786439188987096 us; error 59 us; skew 500 ppm
I20260811 09:06:28.987526 32750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:06:28.990064 32750 webserver.cc:460] Webserver started at http://127.9.40.67:36737/ using document root <none> and password file <none>
I20260811 09:06:28.990814 32750 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:28.990948 32750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:28.991312 32750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:28.995929 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data/instance:
uuid: "b1902c7d23e84deb9160676ad18874fd"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "90202a04ccf5747da249e950d5f84631"
server_key_iv: "657f97d8ccf8354c5aa2b2c08104c729"
server_key_version: "encryptionkey@0"
I20260811 09:06:28.997152 32750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "b1902c7d23e84deb9160676ad18874fd"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "90202a04ccf5747da249e950d5f84631"
server_key_iv: "657f97d8ccf8354c5aa2b2c08104c729"
server_key_version: "encryptionkey@0"
I20260811 09:06:29.003690 32750 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:06:29.008001 300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:29.009572 32750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:29.009783 32750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal
uuid: "b1902c7d23e84deb9160676ad18874fd"
format_stamp: "Formatted at 2026-08-11 09:06:28 on dist-test-slave-cbsf"
server_key: "90202a04ccf5747da249e950d5f84631"
server_key_iv: "657f97d8ccf8354c5aa2b2c08104c729"
server_key_version: "encryptionkey@0"
I20260811 09:06:29.010016 32750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:29.031471 32750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:29.032434 32750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:29.032819 32750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:29.033896 32750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:06:29.035573 32750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:06:29.035703 32750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:29.035841 32750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:06:29.035929 32750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:29.070408 32750 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:36693
I20260811 09:06:29.070533 435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:36693 every 8 connection(s)
I20260811 09:06:29.071772 32750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/data/info.pb
I20260811 09:06:29.074059 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 32750
I20260811 09:06:29.074272 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal/instance
I20260811 09:06:29.074889 9377 external_mini_cluster.cc:1428] Setting key ba0a002ee6df5e578863c37affd26c1b
I20260811 09:06:29.085572 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.074922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43547 (local address 127.9.40.126:42973)
0811 09:06:29.075144 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.075149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.076961 (+ 1812us) server_negotiation.cc:407] Connection header received
0811 09:06:29.078828 (+ 1867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.078842 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.078944 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.079064 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:29.080174 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.081015 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.082656 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.083071 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.083213 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.084621 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:29.084635 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:29.084638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:29.084659 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:29.084737 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:29.084754 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:29.084858 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.085098 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.085208 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 09:06:29.086663 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.075531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:42973 (local address 127.9.40.67:43547)
0811 09:06:29.076769 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.076826 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.078532 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.079308 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.079318 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.079416 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:06:29.080001 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.080018 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.081195 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.081199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.082473 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.082485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.083333 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.083340 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.083587 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.084271 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:06:29.084274 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:06:29.084318 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:06:29.084415 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:06:29.084859 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:06:29.084875 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:06:29.084885 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.085085 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.085532 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":412,"threads_started":1}
I20260811 09:06:29.088388 436 heartbeater.cc:347] Connected to a master server at 127.9.40.126:42973
I20260811 09:06:29.088778 436 heartbeater.cc:464] Registering TS with master...
I20260811 09:06:29.089833 436 heartbeater.cc:511] Master 127.9.40.126:42973 requested a full tablet report, sending...
I20260811 09:06:29.091382 32373 ts_manager.cc:194] Registered new tserver with Master: b1902c7d23e84deb9160676ad18874fd (127.9.40.67:36693)
I20260811 09:06:29.092181 32373 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:43547
I20260811 09:06:29.094183 32593 heartbeater.cc:503] Master 127.9.40.126:42973 was elected leader, sending a full tablet report...
I20260811 09:06:29.102670 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:06:29.112525 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.106758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.9.40.126:42973)
0811 09:06:29.106932 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.106937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.107454 (+ 517us) server_negotiation.cc:407] Connection header received
0811 09:06:29.107598 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.107602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.107667 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.107748 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:29.108217 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.109063 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.110385 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.110819 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.110911 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.111948 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:29.111952 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:29.111955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:29.111973 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:29.112004 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:29.112015 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:29.112084 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.112214 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.112290 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:06:29.116827 32373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:06:29.124042 32373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:06:29.162503 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.137059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.9.40.65:38617)
0811 09:06:29.153863 (+ 16804us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.153875 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.153917 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:06:29.154071 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.154082 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.154366 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.154630 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:29.155750 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.157228 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.160099 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.161179 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.161337 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.161681 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.161949 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.162137 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16524,"thread_start_us":18639,"threads_started":1}
I20260811 09:06:29.162539 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.136815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39384)
0811 09:06:29.149744 (+ 12929us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.149805 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.150078 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.154934 (+ 4856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.154942 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.154994 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:29.155495 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.155512 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.157419 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.157424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.159890 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.159906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.161394 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.161400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.161650 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.161717 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.161889 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.162095 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12623,"mutex_wait_us":27,"thread_start_us":15197,"threads_started":1}
I20260811 09:06:29.162658 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.137563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39667 (local address 127.0.0.1:36302)
0811 09:06:29.152660 (+ 15097us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.152685 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.152860 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.154237 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.154244 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.154272 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:29.154716 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.154731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.156752 (+ 2021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.156758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.159407 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.159421 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.161789 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.161795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.161971 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.162003 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.162096 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.162242 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14955,"mutex_wait_us":1,"thread_start_us":14818,"threads_started":1}
I20260811 09:06:29.163409 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.138268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36302 (local address 127.9.40.66:39667)
0811 09:06:29.153431 (+ 15163us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.153451 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.153484 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:06:29.153610 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.153620 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.153941 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.154926 (+ 985us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:29.155139 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.156565 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.159637 (+ 3072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.161261 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.161404 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.161645 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.162798 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.163022 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15000,"thread_start_us":14875,"threads_started":1}
I20260811 09:06:29.166285 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.137188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36693 (local address 127.0.0.1:47214)
0811 09:06:29.151718 (+ 14530us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.151744 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.151905 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.157219 (+ 5314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.157223 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.157252 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:29.157749 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.157762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.159786 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.159791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.162930 (+ 3139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.162947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.165411 (+ 2464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.165417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.165693 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.165728 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.165832 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.165968 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14339,"thread_start_us":12042,"threads_started":1}
I20260811 09:06:29.166450 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.137943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.9.40.67:36693)
0811 09:06:29.156535 (+ 18592us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.156541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.156570 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:06:29.156680 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.156687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.156979 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.157208 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:29.157986 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.159638 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.163206 (+ 3568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.165181 (+ 1975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.165341 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.165638 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.165894 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.166063 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18428,"thread_start_us":18260,"threads_started":1}
I20260811 09:06:29.167511 32671 tablet_service.cc:1467] Processing CreateTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f (DEFAULT_TABLE table=test-table [id=cc8cf269846845c3aa552f7f9a59666a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:29.167506 32516 tablet_service.cc:1467] Processing CreateTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f (DEFAULT_TABLE table=test-table [id=cc8cf269846845c3aa552f7f9a59666a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:29.169068 32516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8177d057be045dda1ce3c8c3b2fe17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:29.169097 32671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8177d057be045dda1ce3c8c3b2fe17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:29.171192 358 tablet_service.cc:1467] Processing CreateTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f (DEFAULT_TABLE table=test-table [id=cc8cf269846845c3aa552f7f9a59666a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:06:29.172813 358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8177d057be045dda1ce3c8c3b2fe17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:29.183303 455 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Bootstrap starting.
I20260811 09:06:29.185724 456 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Bootstrap starting.
I20260811 09:06:29.188441 455 tablet_bootstrap.cc:654] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:29.189590 457 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Bootstrap starting.
I20260811 09:06:29.189816 456 tablet_bootstrap.cc:654] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:29.190011 455 log.cc:824] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:29.191195 456 log.cc:824] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:29.193547 457 tablet_bootstrap.cc:654] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:29.194664 457 log.cc:824] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:29.194975 455 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: No bootstrap required, opened a new log
I20260811 09:06:29.195360 455 ts_tablet_manager.cc:1397] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.004s
I20260811 09:06:29.199307 456 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: No bootstrap required, opened a new log
I20260811 09:06:29.199628 456 ts_tablet_manager.cc:1397] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260811 09:06:29.199647 457 tablet_bootstrap.cc:492] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: No bootstrap required, opened a new log
I20260811 09:06:29.199922 457 ts_tablet_manager.cc:1397] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 09:06:29.207868 456 raft_consensus.cc:348] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.207899 455 raft_consensus.cc:348] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.208472 456 raft_consensus.cc:374] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:29.208583 456 raft_consensus.cc:729] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0023292904345039282b146ffca3c00, State: Initialized, Role: FOLLOWER
I20260811 09:06:29.208606 455 raft_consensus.cc:374] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:29.208737 455 raft_consensus.cc:729] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ffb5af257c043beaf8eca304fb727c7, State: Initialized, Role: FOLLOWER
I20260811 09:06:29.209317 456 consensus_queue.cc:260] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.209424 455 consensus_queue.cc:260] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.210531 32746 heartbeater.cc:503] Master 127.9.40.126:42973 was elected leader, sending a full tablet report...
I20260811 09:06:29.211483 456 ts_tablet_manager.cc:1428] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 09:06:29.212721 457 raft_consensus.cc:348] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.213266 457 raft_consensus.cc:374] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:29.213441 457 raft_consensus.cc:729] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1902c7d23e84deb9160676ad18874fd, State: Initialized, Role: FOLLOWER
I20260811 09:06:29.214313 455 ts_tablet_manager.cc:1428] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260811 09:06:29.214100 457 consensus_queue.cc:260] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.215665 436 heartbeater.cc:503] Master 127.9.40.126:42973 was elected leader, sending a full tablet report...
I20260811 09:06:29.216436 457 ts_tablet_manager.cc:1428] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Time spent starting tablet: real 0.016s user 0.014s sys 0.004s
I20260811 09:06:29.267838 462 raft_consensus.cc:482] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:06:29.268172 462 raft_consensus.cc:504] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.269948 462 leader_election.cc:288] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1902c7d23e84deb9160676ad18874fd (127.9.40.67:36693), f0023292904345039282b146ffca3c00 (127.9.40.66:39667)
I20260811 09:06:29.279206 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.270520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43833 (local address 127.9.40.66:39667)
0811 09:06:29.270739 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.270746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.271981 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 09:06:29.272142 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.272147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.272219 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.272293 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:29.272915 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.273907 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.276869 (+ 2962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.278146 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.278305 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.278408 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.278783 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.278944 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:06:29.279415 465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.270523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39667 (local address 127.9.40.65:43833)
0811 09:06:29.271859 (+ 1336us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.271875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.271992 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.272347 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.272352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.272374 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:29.272746 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.272762 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.274038 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.274045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.276616 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.276634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.278391 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.278398 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.278611 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.278647 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.278753 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.278914 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":117,"thread_start_us":405,"threads_started":1}
I20260811 09:06:29.279731 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.270586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56201 (local address 127.9.40.67:36693)
0811 09:06:29.270777 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.270784 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.271353 (+ 569us) server_negotiation.cc:407] Connection header received
0811 09:06:29.271668 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.271674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.271773 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.271856 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:06:29.272519 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.273667 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.276933 (+ 3266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.278206 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.278327 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.278417 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.279331 (+ 914us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.279499 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:06:29.280465 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.270472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36693 (local address 127.9.40.65:56201)
0811 09:06:29.271232 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.271249 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:06:29.271445 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:06:29.271913 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:06:29.271920 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:06:29.271960 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:06:29.272364 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.272375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.273826 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.273830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.276616 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:06:29.276634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.278942 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.278951 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:06:29.279111 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.279142 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:06:29.280005 (+ 863us) client_negotiation.cc:240] Negotiation successful
0811 09:06:29.280109 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":156,"thread_start_us":1065,"threads_started":1}
I20260811 09:06:29.280821 32694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8177d057be045dda1ce3c8c3b2fe17f" candidate_uuid: "6ffb5af257c043beaf8eca304fb727c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0023292904345039282b146ffca3c00" is_pre_election: true
I20260811 09:06:29.281461 32694 raft_consensus.cc:2454] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ffb5af257c043beaf8eca304fb727c7 in term 0.
I20260811 09:06:29.281642 380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8177d057be045dda1ce3c8c3b2fe17f" candidate_uuid: "6ffb5af257c043beaf8eca304fb727c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1902c7d23e84deb9160676ad18874fd" is_pre_election: true
I20260811 09:06:29.282194 380 raft_consensus.cc:2454] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ffb5af257c043beaf8eca304fb727c7 in term 0.
I20260811 09:06:29.282244 32462 leader_election.cc:302] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ffb5af257c043beaf8eca304fb727c7, f0023292904345039282b146ffca3c00; no voters:
I20260811 09:06:29.282730 462 raft_consensus.cc:2781] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:06:29.282891 462 raft_consensus.cc:482] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:06:29.283011 462 raft_consensus.cc:3037] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:29.285362 462 raft_consensus.cc:504] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.285982 462 leader_election.cc:288] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [CANDIDATE]: Term 1 election: Requested vote from peers b1902c7d23e84deb9160676ad18874fd (127.9.40.67:36693), f0023292904345039282b146ffca3c00 (127.9.40.66:39667)
I20260811 09:06:29.286552 380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8177d057be045dda1ce3c8c3b2fe17f" candidate_uuid: "6ffb5af257c043beaf8eca304fb727c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1902c7d23e84deb9160676ad18874fd"
I20260811 09:06:29.286687 32694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8177d057be045dda1ce3c8c3b2fe17f" candidate_uuid: "6ffb5af257c043beaf8eca304fb727c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0023292904345039282b146ffca3c00"
I20260811 09:06:29.286813 380 raft_consensus.cc:3037] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:29.286931 32694 raft_consensus.cc:3037] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:29.289314 380 raft_consensus.cc:2454] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ffb5af257c043beaf8eca304fb727c7 in term 1.
I20260811 09:06:29.289449 32694 raft_consensus.cc:2454] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ffb5af257c043beaf8eca304fb727c7 in term 1.
I20260811 09:06:29.289772 32459 leader_election.cc:302] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ffb5af257c043beaf8eca304fb727c7, b1902c7d23e84deb9160676ad18874fd; no voters:
I20260811 09:06:29.290096 462 raft_consensus.cc:2781] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:29.290627 462 raft_consensus.cc:686] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6ffb5af257c043beaf8eca304fb727c7, State: Running, Role: LEADER
I20260811 09:06:29.291175 462 consensus_queue.cc:237] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } }
I20260811 09:06:29.296152 32371 catalog_manager.cc:5495] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ffb5af257c043beaf8eca304fb727c7 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "6ffb5af257c043beaf8eca304fb727c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ffb5af257c043beaf8eca304fb727c7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 } health_report { overall_health: UNKNOWN } } }
I20260811 09:06:29.321440 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.315799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.9.40.65:38617)
0811 09:06:29.315985 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.315989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.316008 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:06:29.316202 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.316205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.316302 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.316427 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:06:29.316838 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.317698 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.319167 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.319618 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.319757 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.320270 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:06:29.320835 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:06:29.321003 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.321086 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.321173 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 09:06:29.322643 32594 tablet.cc:2369] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:06:29.323534 437 tablet.cc:2369] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:06:29.329334 32747 tablet.cc:2369] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:06:29.330161 32694 raft_consensus.cc:1261] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 1 FOLLOWER]: Refusing update from remote peer 6ffb5af257c043beaf8eca304fb727c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:29.330161 380 raft_consensus.cc:1261] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 1 FOLLOWER]: Refusing update from remote peer 6ffb5af257c043beaf8eca304fb727c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:06:29.331189 468 consensus_queue.cc:1035] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0023292904345039282b146ffca3c00" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:29.331596 462 consensus_queue.cc:1035] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1902c7d23e84deb9160676ad18874fd" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:06:29.344355 472 mvcc.cc:205] Tried to move back new op lower bound from 7317254919485104128 to 7317254919340703744. Current Snapshot: MvccSnapshot[applied={T|T < 7317254919485104128}]
I20260811 09:06:29.357824 32372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:29.358124 32372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:06:29.361758 32372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51ba08605dea41e0b662df7ca2e74378: Sending DeleteTablet for 3 replicas of tablet c8177d057be045dda1ce3c8c3b2fe17f
I20260811 09:06:29.362881 32516 tablet_service.cc:1514] Processing DeleteTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:29 UTC) from {username='slave'} at 127.0.0.1:39384
I20260811 09:06:29.363240 358 tablet_service.cc:1514] Processing DeleteTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:29 UTC) from {username='slave'} at 127.0.0.1:47214
I20260811 09:06:29.364004 32671 tablet_service.cc:1514] Processing DeleteTablet for tablet c8177d057be045dda1ce3c8c3b2fe17f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:06:29 UTC) from {username='slave'} at 127.0.0.1:36302
I20260811 09:06:29.364449 483 tablet_replica.cc:330] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: stopping tablet replica
I20260811 09:06:29.365044 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 32440
I20260811 09:06:29.365029 483 raft_consensus.cc:2229] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:06:29.365927 483 raft_consensus.cc:2258] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:29.367419 484 tablet_replica.cc:330] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: stopping tablet replica
I20260811 09:06:29.367905 484 raft_consensus.cc:2229] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:29.368481 484 raft_consensus.cc:2258] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:29.370005 483 ts_tablet_manager.cc:1905] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:29.370652 484 ts_tablet_manager.cc:1905] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:29.375157 485 tablet_replica.cc:330] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: stopping tablet replica
I20260811 09:06:29.375631 485 raft_consensus.cc:2229] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:06:29.376171 485 raft_consensus.cc:2258] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:06:29.377923 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.366113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.9.40.65:38617)
0811 09:06:29.366322 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.366330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.366781 (+ 451us) server_negotiation.cc:407] Connection header received
0811 09:06:29.367222 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.367231 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.367339 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.367445 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:29.368223 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.369431 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.371234 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.371952 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.372087 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.375500 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:29.375520 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:29.375539 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:29.376539 (+ 1000us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:29.376606 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:29.376634 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:29.376816 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.377411 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.377540 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:06:29.378652 484 ts_tablet_manager.cc:1918] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:29.378675 32587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:29.378834 484 log.cc:1196] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-2/wal/wals/c8177d057be045dda1ce3c8c3b2fe17f
I20260811 09:06:29.379413 484 ts_tablet_manager.cc:1939] T c8177d057be045dda1ce3c8c3b2fe17f P b1902c7d23e84deb9160676ad18874fd: Deleting consensus metadata
I20260811 09:06:29.380007 485 ts_tablet_manager.cc:1905] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:06:29.381379 32354 catalog_manager.cc:4841] TS b1902c7d23e84deb9160676ad18874fd (127.9.40.67:36693): tablet c8177d057be045dda1ce3c8c3b2fe17f (table test-table [id=cc8cf269846845c3aa552f7f9a59666a]) successfully deleted
I20260811 09:06:29.388067 485 ts_tablet_manager.cc:1918] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:29.388255 485 log.cc:1196] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-1/wal/wals/c8177d057be045dda1ce3c8c3b2fe17f
I20260811 09:06:29.388826 485 ts_tablet_manager.cc:1939] T c8177d057be045dda1ce3c8c3b2fe17f P f0023292904345039282b146ffca3c00: Deleting consensus metadata
I20260811 09:06:29.390821 32357 catalog_manager.cc:4841] TS f0023292904345039282b146ffca3c00 (127.9.40.66:39667): tablet c8177d057be045dda1ce3c8c3b2fe17f (table test-table [id=cc8cf269846845c3aa552f7f9a59666a]) successfully deleted
I20260811 09:06:29.479938 483 ts_tablet_manager.cc:1918] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:06:29.480296 483 log.cc:1196] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Deleting WAL directory at /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786438852478101-9377-0/minicluster-data/ts-0/wal/wals/c8177d057be045dda1ce3c8c3b2fe17f
I20260811 09:06:29.481362 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 32440
I20260811 09:06:29.481447 483 ts_tablet_manager.cc:1939] T c8177d057be045dda1ce3c8c3b2fe17f P 6ffb5af257c043beaf8eca304fb727c7: Deleting consensus metadata
I20260811 09:06:29.506275 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 32597
W20260811 09:06:29.506274 32355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:38617 (error 108)
I20260811 09:06:29.508410 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.507849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39406)
0811 09:06:29.508084 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.508216 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260811 09:06:29.508677 32355 catalog_manager.cc:4584] TS 6ffb5af257c043beaf8eca304fb727c7 (127.9.40.65:38617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8177d057be045dda1ce3c8c3b2fe17f: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
I20260811 09:06:29.513872 454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.507147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36308 (local address 127.9.40.66:39667)
0811 09:06:29.507355 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.507362 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.507647 (+ 285us) server_negotiation.cc:407] Connection header received
0811 09:06:29.507852 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.507857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.507943 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.508060 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:29.508581 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.509493 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.510708 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.511096 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.511189 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.512342 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:29.512350 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:29.512364 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:29.513045 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:29.513113 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:29.513133 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:29.513219 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.513448 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.513586 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:06:29.514564 32741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:29.562443 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.561960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39408)
0811 09:06:29.562186 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.562271 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 09:06:29.563432 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.563097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39416)
0811 09:06:29.563244 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.563314 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 09:06:29.605659 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 32597
I20260811 09:06:29.624153 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.623509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39426)
0811 09:06:29.623780 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.623911 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 09:06:29.625391 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.624925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39442)
0811 09:06:29.625108 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.625208 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:06:29.630003 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 32750
I20260811 09:06:29.637650 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.630787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47218 (local address 127.9.40.67:36693)
0811 09:06:29.630973 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:29.630977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:29.631039 (+ 62us) server_negotiation.cc:407] Connection header received
0811 09:06:29.631210 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:29.631214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:29.631307 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:29.631398 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:29.631761 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.632502 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.633800 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:29.634218 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:29.634337 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:29.635649 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:29.635664 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:29.635688 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:29.636570 (+ 882us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:29.636652 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:29.636684 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:29.636991 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:29.637100 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 09:06:29.637271 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:06:29.638355 431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:29.706053 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.705510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39450)
0811 09:06:29.705761 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.705879 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260811 09:06:29.707259 449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:29.706861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38617 (local address 127.0.0.1:39460)
0811 09:06:29.707065 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 09:06:29.707143 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 09:06:29.733254 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 32750
I20260811 09:06:29.758049 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 32338
I20260811 09:06:29.759302 32407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:29.875216 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 32338
2026-08-11T09:06:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2988 ms)
[----------] 6 tests from EncryptionPolicyTest (23552 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:30 dist-test-slave-cbsf krb5kdc[497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:30 dist-test-slave-cbsf krb5kdc[497](info): set up 2 sockets
Aug 11 09:06:30 dist-test-slave-cbsf krb5kdc[497](info): commencing operation
krb5kdc: starting...
W20260811 09:06:31.113739 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:31 dist-test-slave-cbsf krb5kdc[497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:31.464398 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:31.822050 511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:31.822377 511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:31.822569 511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:31.830113 511 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:31.830281 511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:31.830370 511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:31.830497 511 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:31.830578 511 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:31.841034 511 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46395
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:31.842514 511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:31.843911 511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:31.856356 517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:31.856338 519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:31.856467 516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:31.857232 511 server_base.cc:1034] running on GCE node
I20260811 09:06:31.858018 511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:31.859560 511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:31.860831 511 hybrid_clock.cc:648] HybridClock initialized: now 1786439191860756 us; error 56 us; skew 500 ppm
I20260811 09:06:31.861245 511 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:31 dist-test-slave-cbsf krb5kdc[497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439191, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:31.864827 511 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:31.866433 511 webserver.cc:460] Webserver started at http://127.0.0.1:35431/ using document root <none> and password file <none>
I20260811 09:06:31.867177 511 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:31.867326 511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:31.867676 511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:31.870967 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "0694bb4ec7a94a839551bb1d0ad6638e"
format_stamp: "Formatted at 2026-08-11 09:06:31 on dist-test-slave-cbsf"
server_key: "8648d845ee57e89a922d74eb10f57658"
server_key_iv: "b5baead1687d751eab9695be0890a244"
server_key_version: "encryptionkey@0"
I20260811 09:06:31.871807 511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "0694bb4ec7a94a839551bb1d0ad6638e"
format_stamp: "Formatted at 2026-08-11 09:06:31 on dist-test-slave-cbsf"
server_key: "8648d845ee57e89a922d74eb10f57658"
server_key_iv: "b5baead1687d751eab9695be0890a244"
server_key_version: "encryptionkey@0"
I20260811 09:06:31.877492 511 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:06:31.881172 526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:31.882530 511 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:06:31.882738 511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "0694bb4ec7a94a839551bb1d0ad6638e"
format_stamp: "Formatted at 2026-08-11 09:06:31 on dist-test-slave-cbsf"
server_key: "8648d845ee57e89a922d74eb10f57658"
server_key_iv: "b5baead1687d751eab9695be0890a244"
server_key_version: "encryptionkey@0"
I20260811 09:06:31.882992 511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:31.901209 511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:31.906114 511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:31.906440 511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:31.926534 511 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46395
I20260811 09:06:31.926573 582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46395 every 8 connection(s)
I20260811 09:06:31.928001 511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:31.930382 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 511
I20260811 09:06:31.930619 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:31.931320 9377 external_mini_cluster.cc:1428] Setting key ac62f26fc47dc2b0b8075ec13adf5c72
I20260811 09:06:31.932085 583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:31.946928 583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e: Bootstrap starting.
Aug 11 09:06:31 dist-test-slave-cbsf krb5kdc[497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:06:31.950968 583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:31.952409 583 log.cc:824] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:31.956245 583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e: No bootstrap required, opened a new log
I20260811 09:06:31.957938 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:31.933472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40475 (local address 127.0.0.1:46395)
0811 09:06:31.934343 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:31.934359 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:31.934411 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:06:31.937196 (+ 2785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:31.937230 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:31.937830 (+ 600us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:31.938325 (+ 495us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:31.939780 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:31.941780 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:31.943162 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:31.944292 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:31.944502 (+ 210us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:31.948840 (+ 4338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:31.948853 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:31.948876 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:31.948935 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:31.952991 (+ 4056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:31.954099 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:31.954108 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:31.954119 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:31.954331 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:31.954762 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:31.954766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:31.954769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:31.955509 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:31.955862 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:31.956160 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:06:31.956518 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":365,"threads_started":1}
I20260811 09:06:31.962086 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:31.966589 583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260811 09:06:31.967125 583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:31.967249 583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0694bb4ec7a94a839551bb1d0ad6638e, State: Initialized, Role: FOLLOWER
I20260811 09:06:31.967957 583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260811 09:06:31.968266 583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:31.968493 583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:31.968732 583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:31.971994 583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260811 09:06:31.972671 583 leader_election.cc:302] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0694bb4ec7a94a839551bb1d0ad6638e; no voters:
I20260811 09:06:31.973415 583 leader_election.cc:288] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:31.973573 594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:31.974267 594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [term 1 LEADER]: Becoming Leader. State: Replica: 0694bb4ec7a94a839551bb1d0ad6638e, State: Running, Role: LEADER
I20260811 09:06:31.974963 594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } }
I20260811 09:06:31.977638 597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260811 09:06:31.978016 597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:31.978425 583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:31.981134 595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0694bb4ec7a94a839551bb1d0ad6638e. Latest consensus state: current_term: 1 leader_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0694bb4ec7a94a839551bb1d0ad6638e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46395 } } }
I20260811 09:06:31.981606 595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:31.983337 599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:31.990109 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:31.968159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53145 (local address 127.0.0.1:46395)
0811 09:06:31.968410 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:31.968420 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:31.968915 (+ 495us) server_negotiation.cc:407] Connection header received
0811 09:06:31.969130 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:31.969134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:31.969200 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:31.969282 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:31.971205 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:31.971969 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:31.973883 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:31.974908 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:31.975069 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:31.979113 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:31.979123 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:31.979131 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:31.979179 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:31.983966 (+ 4787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:31.985292 (+ 1326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:31.985300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:31.985304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:31.985475 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:31.986210 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:31.986218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:31.986228 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:31.986947 (+ 719us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:31.988983 (+ 2036us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:31.989245 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 09:06:31.989518 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 09:06:31.990211 599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:32.008790 599 catalog_manager.cc:1347] Generated new cluster ID: 23daaccce9294dab951c05f00d5aab31
I20260811 09:06:32.008927 599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:32.026000 599 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:32.027980 599 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:32.036613 599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0694bb4ec7a94a839551bb1d0ad6638e: Generated new TSK 0
I20260811 09:06:32.037575 599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:32.045462 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 511
I20260811 09:06:32.046881 577 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:32.140260 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 511
W20260811 09:06:32.164923 492 connection.cc:535] client connection to 127.0.0.1:46395 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46395): Connection reset by peer (error 104)
2026-08-11T09:06:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2259 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:32 dist-test-slave-cbsf krb5kdc[619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:32 dist-test-slave-cbsf krb5kdc[619](info): set up 2 sockets
Aug 11 09:06:32 dist-test-slave-cbsf krb5kdc[619](info): commencing operation
krb5kdc: starting...
W20260811 09:06:33.377113 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.006s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:33 dist-test-slave-cbsf krb5kdc[619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:33.732620 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45463
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:34.086822 634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:34.087179 634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:34.087337 634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:34.094761 634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:34.094929 634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:34.095045 634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:34.095150 634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:34.095227 634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:34.105720 634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45463
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:34.107182 634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:34.108521 634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:34.121451 642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:34.121457 645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:34.121345 643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:34.122592 634 server_base.cc:1034] running on GCE node
I20260811 09:06:34.123457 634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:34.125010 634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:34.126232 634 hybrid_clock.cc:648] HybridClock initialized: now 1786439194126158 us; error 57 us; skew 500 ppm
I20260811 09:06:34.126744 634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:34 dist-test-slave-cbsf krb5kdc[619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439194, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:34.135834 634 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:34.137866 634 webserver.cc:460] Webserver started at http://127.0.0.1:39107/ using document root <none> and password file <none>
I20260811 09:06:34.138669 634 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:34.138797 634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:34.139125 634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:34.142692 634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "ef07fffb91cb459b8bcf54287d9f7983"
format_stamp: "Formatted at 2026-08-11 09:06:34 on dist-test-slave-cbsf"
server_key: "5cd3c4fbcec8c8bc0e461b1fa40efa5c"
server_key_iv: "77cb68e191e60e9c97569a0942e0df31"
server_key_version: "encryptionkey@0"
I20260811 09:06:34.143545 634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "ef07fffb91cb459b8bcf54287d9f7983"
format_stamp: "Formatted at 2026-08-11 09:06:34 on dist-test-slave-cbsf"
server_key: "5cd3c4fbcec8c8bc0e461b1fa40efa5c"
server_key_iv: "77cb68e191e60e9c97569a0942e0df31"
server_key_version: "encryptionkey@0"
I20260811 09:06:34.149686 634 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260811 09:06:34.153365 654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:34.154821 634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:34.155055 634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "ef07fffb91cb459b8bcf54287d9f7983"
format_stamp: "Formatted at 2026-08-11 09:06:34 on dist-test-slave-cbsf"
server_key: "5cd3c4fbcec8c8bc0e461b1fa40efa5c"
server_key_iv: "77cb68e191e60e9c97569a0942e0df31"
server_key_version: "encryptionkey@0"
I20260811 09:06:34.155284 634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:34.181155 634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:34.185607 634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:34.185933 634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:34.205679 634 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45463
I20260811 09:06:34.205714 713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45463 every 8 connection(s)
I20260811 09:06:34.207166 634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:34.207842 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 634
I20260811 09:06:34.208070 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:34.208738 9377 external_mini_cluster.cc:1428] Setting key 76f9eed1e4e2e296246c31358e24d076
I20260811 09:06:34.211647 714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:34.225927 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:34.211111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48673 (local address 127.0.0.1:45463)
0811 09:06:34.212117 (+ 1006us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:34.212133 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:34.212521 (+ 388us) server_negotiation.cc:407] Connection header received
0811 09:06:34.214862 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:34.214892 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:34.215461 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:34.215936 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:34.216488 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:34.218687 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:34.220296 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:34.221382 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:34.221568 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:34.222932 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:34.222943 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:34.222962 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:34.223767 (+ 805us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:34.223865 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:34.223897 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:34.224078 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:34.224344 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 09:06:34.224545 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":436,"threads_started":1}
I20260811 09:06:34.227072 714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983: Bootstrap starting.
I20260811 09:06:34.230131 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:34.230965 714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:34.232234 714 log.cc:824] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:34.235903 714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983: No bootstrap required, opened a new log
I20260811 09:06:34.242898 717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:34.235623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40345 (local address 127.0.0.1:45463)
0811 09:06:34.235817 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:34.235823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:34.236279 (+ 456us) server_negotiation.cc:407] Connection header received
0811 09:06:34.236464 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:34.236475 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:34.236570 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:34.236684 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:34.237287 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:34.238550 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:34.240088 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:34.240798 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:34.240944 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:34.242069 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:34.242076 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:34.242080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:34.242101 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:34.242169 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:34.242190 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:34.242322 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:34.242484 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 09:06:34.242546 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:06:34.246006 714 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260811 09:06:34.246397 714 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:34.246479 714 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef07fffb91cb459b8bcf54287d9f7983, State: Initialized, Role: FOLLOWER
I20260811 09:06:34.247045 714 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260811 09:06:34.247254 714 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:34.247354 714 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:34.247493 714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:34.250221 714 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260811 09:06:34.250710 714 leader_election.cc:302] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef07fffb91cb459b8bcf54287d9f7983; no voters:
I20260811 09:06:34.251307 714 leader_election.cc:288] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:34.251477 724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:34.252246 724 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [term 1 LEADER]: Becoming Leader. State: Replica: ef07fffb91cb459b8bcf54287d9f7983, State: Running, Role: LEADER
I20260811 09:06:34.252851 724 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } }
I20260811 09:06:34.253608 714 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:34.255620 725 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef07fffb91cb459b8bcf54287d9f7983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } } }
I20260811 09:06:34.256072 725 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:34.255995 726 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef07fffb91cb459b8bcf54287d9f7983. Latest consensus state: current_term: 1 leader_uuid: "ef07fffb91cb459b8bcf54287d9f7983" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef07fffb91cb459b8bcf54287d9f7983" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45463 } } }
I20260811 09:06:34.256312 726 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:34.257764 732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:34.264882 732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:34.274380 732 catalog_manager.cc:1347] Generated new cluster ID: 10693be821a44403baa23ed0c90f1218
I20260811 09:06:34.274513 732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:34.288684 732 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:34.290160 732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:34.299799 732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef07fffb91cb459b8bcf54287d9f7983: Generated new TSK 0
I20260811 09:06:34.300679 732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:34.308061 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 634
W20260811 09:06:34.309312 709 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48673
W20260811 09:06:34.310266 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 634: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:34.310467 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 634
W20260811 09:06:34.335819 615 connection.cc:535] client connection to 127.0.0.1:45463 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45463): Connection reset by peer (error 104)
2026-08-11T09:06:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2170 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:34 dist-test-slave-cbsf krb5kdc[748](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:34 dist-test-slave-cbsf krb5kdc[748](info): set up 2 sockets
Aug 11 09:06:34 dist-test-slave-cbsf krb5kdc[748](info): commencing operation
krb5kdc: starting...
W20260811 09:06:35.540957 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:35 dist-test-slave-cbsf krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:35.881590 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:36.237044 764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:36.237370 764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:36.237466 764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:36.244830 764 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:36.244952 764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:36.245007 764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:36.245069 764 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:36.245114 764 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:36.255434 764 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:36.256774 764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:36.258068 764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:36.270561 770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:36.270605 771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:36.270658 775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:36.271456 764 server_base.cc:1034] running on GCE node
I20260811 09:06:36.272238 764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:36.273753 764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:36.274955 764 hybrid_clock.cc:648] HybridClock initialized: now 1786439196274907 us; error 36 us; skew 500 ppm
I20260811 09:06:36.275278 764 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:36 dist-test-slave-cbsf krb5kdc[748](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439196, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:36.279031 764 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:36.280633 764 webserver.cc:460] Webserver started at http://127.0.0.1:44313/ using document root <none> and password file <none>
I20260811 09:06:36.281374 764 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:36.281483 764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:36.281759 764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:36.284863 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "53977bff49ea4578aa83733019c83522"
format_stamp: "Formatted at 2026-08-11 09:06:36 on dist-test-slave-cbsf"
server_key: "6f172fe23eb9215117ff4ea7f7b5e85a"
server_key_iv: "671cecc996c1ff0fa746b696b896b8b1"
server_key_version: "encryptionkey@0"
I20260811 09:06:36.285593 764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "53977bff49ea4578aa83733019c83522"
format_stamp: "Formatted at 2026-08-11 09:06:36 on dist-test-slave-cbsf"
server_key: "6f172fe23eb9215117ff4ea7f7b5e85a"
server_key_iv: "671cecc996c1ff0fa746b696b896b8b1"
server_key_version: "encryptionkey@0"
I20260811 09:06:36.290829 764 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:06:36.294335 785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:36.295750 764 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:06:36.295934 764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "53977bff49ea4578aa83733019c83522"
format_stamp: "Formatted at 2026-08-11 09:06:36 on dist-test-slave-cbsf"
server_key: "6f172fe23eb9215117ff4ea7f7b5e85a"
server_key_iv: "671cecc996c1ff0fa746b696b896b8b1"
server_key_version: "encryptionkey@0"
I20260811 09:06:36.296154 764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:36.302002 764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:36.306092 764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:36.306356 764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:36.326025 764 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45319
I20260811 09:06:36.326081 840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45319 every 8 connection(s)
I20260811 09:06:36.327574 764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:36.331650 841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:36.335850 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 764
I20260811 09:06:36.336041 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:36.336522 9377 external_mini_cluster.cc:1428] Setting key 453d05c814930b7b3dd5648ddd9fc270
I20260811 09:06:36.344666 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:36.338446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60777 (local address 127.0.0.1:45319)
0811 09:06:36.339317 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:36.339329 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:36.339384 (+ 55us) server_negotiation.cc:407] Connection header received
0811 09:06:36.341119 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:36.341146 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:36.341519 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:36.341934 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:36.343066 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:36.343073 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:36.343089 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:36.343111 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:36.343164 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:36.343187 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:36.343427 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:36.343565 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 09:06:36.343749 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":373,"threads_started":1}
I20260811 09:06:36.345314 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522: Bootstrap starting.
I20260811 09:06:36.348515 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:36.348999 841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:36.350044 841 log.cc:824] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:36.353646 841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522: No bootstrap required, opened a new log
I20260811 09:06:36.355818 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:36.353266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50347 (local address 127.0.0.1:45319)
0811 09:06:36.353450 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:36.353457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:36.353792 (+ 335us) server_negotiation.cc:407] Connection header received
0811 09:06:36.354032 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:36.354040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:36.354119 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:36.354241 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:36.355156 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:36.355161 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:36.355166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:36.355183 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:36.355231 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:36.355256 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:36.355380 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:36.355452 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 09:06:36.355513 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:06:36.362185 841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260811 09:06:36.362684 841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:36.362787 841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53977bff49ea4578aa83733019c83522, State: Initialized, Role: FOLLOWER
I20260811 09:06:36.363399 841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260811 09:06:36.363652 841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:36.363816 841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:36.363994 841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:36.366348 841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260811 09:06:36.366927 841 leader_election.cc:302] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53977bff49ea4578aa83733019c83522; no voters:
I20260811 09:06:36.367483 841 leader_election.cc:288] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:36.367658 851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:36.368386 851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [term 1 LEADER]: Becoming Leader. State: Replica: 53977bff49ea4578aa83733019c83522, State: Running, Role: LEADER
I20260811 09:06:36.368994 851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260811 09:06:36.369849 841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:36.371980 853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53977bff49ea4578aa83733019c83522. Latest consensus state: current_term: 1 leader_uuid: "53977bff49ea4578aa83733019c83522" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260811 09:06:36.372397 853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:36.372143 852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53977bff49ea4578aa83733019c83522" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53977bff49ea4578aa83733019c83522" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260811 09:06:36.372543 852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:36.375017 859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:36.380798 859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:36.389945 859 catalog_manager.cc:1347] Generated new cluster ID: e8747d43f43644a7848aff3e1bc898ae
I20260811 09:06:36.390122 859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:36.445626 859 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:36.446189 859 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:36.453724 859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53977bff49ea4578aa83733019c83522: Generated new TSK 0
I20260811 09:06:36.454690 859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:36.472718 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 764
W20260811 09:06:36.473881 836 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60777
W20260811 09:06:36.474695 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 764: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:36.474843 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 764
2026-08-11T09:06:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2162 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:36 dist-test-slave-cbsf krb5kdc[874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:36 dist-test-slave-cbsf krb5kdc[874](info): set up 2 sockets
Aug 11 09:06:36 dist-test-slave-cbsf krb5kdc[874](info): commencing operation
krb5kdc: starting...
W20260811 09:06:37.709928 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.004s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:37 dist-test-slave-cbsf krb5kdc[874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:38.060606 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:38.417016 888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:38.417353 888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:38.417542 888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:38.425448 888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:38.425575 888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:38.425704 888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:38.425813 888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:38.425889 888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:38.436488 888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:38.437956 888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:38.439390 888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:38.451548 893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:38.451670 894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:38.451936 896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:38.452929 888 server_base.cc:1034] running on GCE node
I20260811 09:06:38.453832 888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:38.455325 888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:38.456538 888 hybrid_clock.cc:648] HybridClock initialized: now 1786439198456484 us; error 38 us; skew 500 ppm
I20260811 09:06:38.456943 888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:38 dist-test-slave-cbsf krb5kdc[874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439198, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:38.460453 888 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:38.462201 888 webserver.cc:460] Webserver started at http://127.0.0.1:41459/ using document root <none> and password file <none>
I20260811 09:06:38.462945 888 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:38.463090 888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:38.463425 888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:38.466626 888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "69c57fb8431347369e1123a102978a05"
format_stamp: "Formatted at 2026-08-11 09:06:38 on dist-test-slave-cbsf"
server_key: "1313b593439d042d5d54696f7b073270"
server_key_iv: "39cb301c03774fe465fa248a70d71b49"
server_key_version: "encryptionkey@0"
I20260811 09:06:38.467412 888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "69c57fb8431347369e1123a102978a05"
format_stamp: "Formatted at 2026-08-11 09:06:38 on dist-test-slave-cbsf"
server_key: "1313b593439d042d5d54696f7b073270"
server_key_iv: "39cb301c03774fe465fa248a70d71b49"
server_key_version: "encryptionkey@0"
I20260811 09:06:38.472970 888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 09:06:38.476588 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:38.477993 888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:38.478199 888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "69c57fb8431347369e1123a102978a05"
format_stamp: "Formatted at 2026-08-11 09:06:38 on dist-test-slave-cbsf"
server_key: "1313b593439d042d5d54696f7b073270"
server_key_iv: "39cb301c03774fe465fa248a70d71b49"
server_key_version: "encryptionkey@0"
I20260811 09:06:38.478406 888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:38.506906 888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:38.511215 888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:38.511538 888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:38.531669 888 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41239
I20260811 09:06:38.531716 965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41239 every 8 connection(s)
I20260811 09:06:38.533206 888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:38.535410 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 888
I20260811 09:06:38.535647 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:38.536120 9377 external_mini_cluster.cc:1428] Setting key 39399fb969b72e07777e4345512d185a
I20260811 09:06:38.537468 966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:06:38 dist-test-slave-cbsf krb5kdc[874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:06:38.552155 966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05: Bootstrap starting.
I20260811 09:06:38.556561 966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:38.557983 966 log.cc:824] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:38.561609 966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05: No bootstrap required, opened a new log
I20260811 09:06:38.561820 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:38.538023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53629 (local address 127.0.0.1:41239)
0811 09:06:38.538847 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:38.538863 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:38.538913 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:06:38.541065 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:38.541101 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:38.541631 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:38.542098 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:38.543427 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:38.545576 (+ 2149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:38.546861 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:38.548002 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:38.548219 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:38.552810 (+ 4591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:38.552830 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:38.552851 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:38.552921 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:38.557199 (+ 4278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:38.558000 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:38.558010 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:38.558020 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:38.558236 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:38.558715 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:38.558723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:38.558729 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:38.559318 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:38.559653 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:38.560038 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 09:06:38.560472 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":326,"threads_started":1}
I20260811 09:06:38.565759 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:38.570079 966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } }
I20260811 09:06:38.570565 966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:38.570705 966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69c57fb8431347369e1123a102978a05, State: Initialized, Role: FOLLOWER
I20260811 09:06:38.571375 966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } }
I20260811 09:06:38.571643 966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:38.571803 966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:38.571990 966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:38.574411 966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } }
I20260811 09:06:38.574944 966 leader_election.cc:302] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69c57fb8431347369e1123a102978a05; no voters:
I20260811 09:06:38.575663 966 leader_election.cc:288] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:38.575819 976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:38.576742 976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [term 1 LEADER]: Becoming Leader. State: Replica: 69c57fb8431347369e1123a102978a05, State: Running, Role: LEADER
I20260811 09:06:38.577447 976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } }
I20260811 09:06:38.578148 966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:38.581774 980 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69c57fb8431347369e1123a102978a05. Latest consensus state: current_term: 1 leader_uuid: "69c57fb8431347369e1123a102978a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } } }
I20260811 09:06:38.582285 980 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:38.583544 986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:38.583431 977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69c57fb8431347369e1123a102978a05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69c57fb8431347369e1123a102978a05" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41239 } } }
I20260811 09:06:38.583911 977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:38.591203 986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:38.597002 969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:38.570368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41895 (local address 127.0.0.1:41239)
0811 09:06:38.570579 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:38.570587 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:38.571099 (+ 512us) server_negotiation.cc:407] Connection header received
0811 09:06:38.571304 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:38.571316 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:38.571383 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:38.571488 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:38.573087 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:38.573852 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:38.575178 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:38.575689 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:38.575841 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:38.578795 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:38.578803 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:38.578809 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:38.578852 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:38.582763 (+ 3911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:38.583691 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:38.583702 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:38.583708 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:38.583901 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:38.584419 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:38.584426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:38.584430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:38.584865 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:38.585162 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:38.586592 (+ 1430us) server_negotiation.cc:299] Negotiation successful
0811 09:06:38.586906 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:06:38.606885 986 catalog_manager.cc:1347] Generated new cluster ID: 2df74d1593cb44fcb16c149dd2311dd1
I20260811 09:06:38.607056 986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:38.629478 986 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:38.631531 986 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:38.645246 986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69c57fb8431347369e1123a102978a05: Generated new TSK 0
I20260811 09:06:38.646409 986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:38.655470 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 888
I20260811 09:06:38.656643 960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:38.754091 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 888
2026-08-11T09:06:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:38 dist-test-slave-cbsf krb5kdc[1002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:38 dist-test-slave-cbsf krb5kdc[1002](info): set up 2 sockets
Aug 11 09:06:38 dist-test-slave-cbsf krb5kdc[1002](info): commencing operation
krb5kdc: starting...
W20260811 09:06:39.993541 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:40 dist-test-slave-cbsf krb5kdc[1002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:40.354681 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45317
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:40.718856 1017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:40.719228 1017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:40.719411 1017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:40.726768 1017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:40.726956 1017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:40.727078 1017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:40.727200 1017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:40.727288 1017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:40.737787 1017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:40.739224 1017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:40.740603 1017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:40.753986 1022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:40.753906 1025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:40.753955 1023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:40.754628 1017 server_base.cc:1034] running on GCE node
I20260811 09:06:40.755539 1017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:40.757236 1017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:40.758518 1017 hybrid_clock.cc:648] HybridClock initialized: now 1786439200758423 us; error 71 us; skew 500 ppm
I20260811 09:06:40.758908 1017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:40 dist-test-slave-cbsf krb5kdc[1002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439200, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:40.767684 1017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:40.769587 1017 webserver.cc:460] Webserver started at http://127.0.0.1:36505/ using document root <none> and password file <none>
I20260811 09:06:40.770354 1017 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:40.770519 1017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:40.770852 1017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:40.774108 1017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "93744b5ccf9c414589411e637ca036a7"
format_stamp: "Formatted at 2026-08-11 09:06:40 on dist-test-slave-cbsf"
server_key: "91d3e1dedac72df0722b0003208ad05d"
server_key_iv: "988b42df6000257a4c785f849fb31ac4"
server_key_version: "encryptionkey@0"
I20260811 09:06:40.774930 1017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "93744b5ccf9c414589411e637ca036a7"
format_stamp: "Formatted at 2026-08-11 09:06:40 on dist-test-slave-cbsf"
server_key: "91d3e1dedac72df0722b0003208ad05d"
server_key_iv: "988b42df6000257a4c785f849fb31ac4"
server_key_version: "encryptionkey@0"
I20260811 09:06:40.780570 1017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:06:40.784250 1033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:40.785737 1017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:40.785930 1017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "93744b5ccf9c414589411e637ca036a7"
format_stamp: "Formatted at 2026-08-11 09:06:40 on dist-test-slave-cbsf"
server_key: "91d3e1dedac72df0722b0003208ad05d"
server_key_iv: "988b42df6000257a4c785f849fb31ac4"
server_key_version: "encryptionkey@0"
I20260811 09:06:40.786157 1017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:40.809607 1017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:40.814500 1017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:40.814841 1017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:40.835616 1017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45317
I20260811 09:06:40.835654 1093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45317 every 8 connection(s)
I20260811 09:06:40.837222 1017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:40.841193 1094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:40.841872 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1017
I20260811 09:06:40.842082 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:40.842550 9377 external_mini_cluster.cc:1428] Setting key bbf9cbf4f0ed07da58012a290aa0fa77
I20260811 09:06:40.856935 1094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7: Bootstrap starting.
I20260811 09:06:40.859141 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:40.844655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39351 (local address 127.0.0.1:45317)
0811 09:06:40.845581 (+ 926us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:40.845598 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:40.845664 (+ 66us) server_negotiation.cc:407] Connection header received
0811 09:06:40.848391 (+ 2727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:40.848425 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:40.849072 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:40.849573 (+ 501us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:40.850103 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:40.852033 (+ 1930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:40.853536 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:40.854685 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:40.854867 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:40.856054 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:40.856067 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:40.856088 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:40.856881 (+ 793us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:40.856995 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:40.857030 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:40.857167 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:40.857430 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 09:06:40.857722 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":362,"threads_started":1}
I20260811 09:06:40.860900 1094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:40.862118 1094 log.cc:824] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:40.863375 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:40.865407 1094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7: No bootstrap required, opened a new log
I20260811 09:06:40.875424 1097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:40.868235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37627 (local address 127.0.0.1:45317)
0811 09:06:40.868499 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:40.868508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:40.869027 (+ 519us) server_negotiation.cc:407] Connection header received
0811 09:06:40.869241 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:40.869245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:40.869313 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:40.869401 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:40.870060 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:40.870865 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:40.872834 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:40.873320 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:40.873424 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:40.874654 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:40.874659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:40.874663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:40.874681 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:40.874723 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:40.874739 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:40.874812 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:40.874978 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:06:40.875063 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:06:40.875387 1094 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } }
I20260811 09:06:40.875921 1094 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:40.876078 1094 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93744b5ccf9c414589411e637ca036a7, State: Initialized, Role: FOLLOWER
I20260811 09:06:40.876765 1094 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } }
I20260811 09:06:40.877049 1094 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:40.877228 1094 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:40.877405 1094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:40.879688 1094 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } }
I20260811 09:06:40.880231 1094 leader_election.cc:302] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93744b5ccf9c414589411e637ca036a7; no voters:
I20260811 09:06:40.880980 1094 leader_election.cc:288] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:40.881095 1107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:40.881840 1107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [term 1 LEADER]: Becoming Leader. State: Replica: 93744b5ccf9c414589411e637ca036a7, State: Running, Role: LEADER
I20260811 09:06:40.882526 1107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } }
I20260811 09:06:40.883752 1094 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:40.885641 1108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93744b5ccf9c414589411e637ca036a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } } }
I20260811 09:06:40.886044 1108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:40.886848 1109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93744b5ccf9c414589411e637ca036a7. Latest consensus state: current_term: 1 leader_uuid: "93744b5ccf9c414589411e637ca036a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93744b5ccf9c414589411e637ca036a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45317 } } }
I20260811 09:06:40.887233 1109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:40.888782 1113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:40.896925 1113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:40.906919 1113 catalog_manager.cc:1347] Generated new cluster ID: e48c62fb5c3149b9ba018a43c00a1b05
I20260811 09:06:40.907065 1113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:40.929126 1113 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:40.930491 1113 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:40.941753 1113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93744b5ccf9c414589411e637ca036a7: Generated new TSK 0
I20260811 09:06:40.942685 1113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:40.949482 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1017
W20260811 09:06:40.951143 1089 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39351
W20260811 09:06:40.951928 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1017: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:40.952099 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1017
2026-08-11T09:06:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2199 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:41 dist-test-slave-cbsf krb5kdc[1130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:41 dist-test-slave-cbsf krb5kdc[1130](info): set up 2 sockets
Aug 11 09:06:41 dist-test-slave-cbsf krb5kdc[1130](info): commencing operation
krb5kdc: starting...
W20260811 09:06:42.189379 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.007s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:42 dist-test-slave-cbsf krb5kdc[1130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:42.552464 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46421
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46421
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:42.913178 1143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:42.913487 1143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:42.913584 1143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:42.921196 1143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:42.921326 1143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:42.921373 1143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:42.921439 1143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:42.921484 1143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:42.931744 1143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:42.933101 1143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:42.934372 1143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:42.947034 1149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:42.947094 1150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:42.947223 1152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:42.948292 1143 server_base.cc:1034] running on GCE node
I20260811 09:06:42.949200 1143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:42.950680 1143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:42.951925 1143 hybrid_clock.cc:648] HybridClock initialized: now 1786439202951879 us; error 39 us; skew 500 ppm
I20260811 09:06:42.952316 1143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:42 dist-test-slave-cbsf krb5kdc[1130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439202, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:42.955565 1143 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:42.957265 1143 webserver.cc:460] Webserver started at http://127.0.0.1:39299/ using document root <none> and password file <none>
I20260811 09:06:42.957988 1143 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:42.958137 1143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:42.958447 1143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:42.961572 1143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "2aaecc48022c4e9c8b260e80c1432b1d"
format_stamp: "Formatted at 2026-08-11 09:06:42 on dist-test-slave-cbsf"
server_key: "1744cc145258665eded095571bbca920"
server_key_iv: "24f7104edb55ec1ea4977838b559ba92"
server_key_version: "encryptionkey@0"
I20260811 09:06:42.962384 1143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2aaecc48022c4e9c8b260e80c1432b1d"
format_stamp: "Formatted at 2026-08-11 09:06:42 on dist-test-slave-cbsf"
server_key: "1744cc145258665eded095571bbca920"
server_key_iv: "24f7104edb55ec1ea4977838b559ba92"
server_key_version: "encryptionkey@0"
I20260811 09:06:42.967859 1143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:06:42.971509 1161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:42.972940 1143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:06:42.973151 1143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "2aaecc48022c4e9c8b260e80c1432b1d"
format_stamp: "Formatted at 2026-08-11 09:06:42 on dist-test-slave-cbsf"
server_key: "1744cc145258665eded095571bbca920"
server_key_iv: "24f7104edb55ec1ea4977838b559ba92"
server_key_version: "encryptionkey@0"
I20260811 09:06:42.973395 1143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:42.979671 1143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:42.984050 1143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:42.984372 1143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:43.007006 1143 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46421
I20260811 09:06:43.007050 1219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46421 every 8 connection(s)
I20260811 09:06:43.008522 1143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:43.012388 1220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:43.018250 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1143
I20260811 09:06:43.018455 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:43.019040 9377 external_mini_cluster.cc:1428] Setting key 3d6ee63e78724c74f4fabf7d3196830a
I20260811 09:06:43.024135 1220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d: Bootstrap starting.
I20260811 09:06:43.027375 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:43.021088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34733 (local address 127.0.0.1:46421)
0811 09:06:43.022000 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:43.022013 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:43.022056 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:06:43.023785 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:43.023809 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:43.024147 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:43.024635 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:43.025768 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:43.025775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:43.025788 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:43.025804 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:43.025872 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:43.025892 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:43.026141 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:43.026287 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 09:06:43.026482 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":439,"threads_started":1}
I20260811 09:06:43.027733 1220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:43.028991 1220 log.cc:824] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:43.031538 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:43.032346 1220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d: No bootstrap required, opened a new log
I20260811 09:06:43.039991 1220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } }
I20260811 09:06:43.040422 1220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:43.040570 1220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aaecc48022c4e9c8b260e80c1432b1d, State: Initialized, Role: FOLLOWER
I20260811 09:06:43.041460 1220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } }
I20260811 09:06:43.041729 1220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:43.041903 1220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:43.042109 1220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:43.044399 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:43.041154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48659 (local address 127.0.0.1:46421)
0811 09:06:43.041359 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:43.041367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:43.042277 (+ 910us) server_negotiation.cc:407] Connection header received
0811 09:06:43.042602 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:43.042607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:43.042659 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:43.042746 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:43.043718 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:43.043722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:43.043726 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:43.043744 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:43.043789 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:43.043804 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:43.043916 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:43.044020 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 09:06:43.044071 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:06:43.045507 1220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } }
I20260811 09:06:43.046164 1220 leader_election.cc:302] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aaecc48022c4e9c8b260e80c1432b1d; no voters:
I20260811 09:06:43.046855 1220 leader_election.cc:288] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:43.046993 1231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:43.047740 1231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [term 1 LEADER]: Becoming Leader. State: Replica: 2aaecc48022c4e9c8b260e80c1432b1d, State: Running, Role: LEADER
I20260811 09:06:43.048341 1231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } }
I20260811 09:06:43.049136 1220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:43.051468 1233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aaecc48022c4e9c8b260e80c1432b1d. Latest consensus state: current_term: 1 leader_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } } }
I20260811 09:06:43.051880 1233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:43.054025 1239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:43.054489 1232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aaecc48022c4e9c8b260e80c1432b1d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46421 } } }
I20260811 09:06:43.054893 1232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:43.060084 1239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:43.070163 1239 catalog_manager.cc:1347] Generated new cluster ID: 53cbd0843473459c8cac21875a5ca79d
I20260811 09:06:43.070297 1239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:43.083261 1239 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:43.083791 1239 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:43.092990 1239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aaecc48022c4e9c8b260e80c1432b1d: Generated new TSK 0
I20260811 09:06:43.093901 1239 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:43.108367 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1143
W20260811 09:06:43.109687 1215 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34733
W20260811 09:06:43.110531 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1143: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:43.110729 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1143
2026-08-11T09:06:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:43 dist-test-slave-cbsf krb5kdc[1256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:43 dist-test-slave-cbsf krb5kdc[1256](info): set up 2 sockets
Aug 11 09:06:43 dist-test-slave-cbsf krb5kdc[1256](info): commencing operation
krb5kdc: starting...
W20260811 09:06:44.333261 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.004s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:44 dist-test-slave-cbsf krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:44.688839 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:45.064486 1271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:45.064893 1271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:45.065073 1271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:45.072515 1271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:45.072693 1271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:45.072852 1271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:45.072948 1271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:45.073041 1271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:45.083338 1271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:45.084968 1271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:45.086373 1271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:45.101029 1276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:45.101090 1277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:45.101090 1280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:45.102169 1271 server_base.cc:1034] running on GCE node
I20260811 09:06:45.102988 1271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:45.104390 1271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:45.105623 1271 hybrid_clock.cc:648] HybridClock initialized: now 1786439205105562 us; error 56 us; skew 500 ppm
I20260811 09:06:45.106050 1271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:45 dist-test-slave-cbsf krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439205, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:45.109380 1271 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:45.110901 1271 webserver.cc:460] Webserver started at http://127.0.0.1:45069/ using document root <none> and password file <none>
I20260811 09:06:45.111635 1271 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:45.111788 1271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:45.112111 1271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:45.115242 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "d87a866ab7c048beb25a3ee166c88f56"
format_stamp: "Formatted at 2026-08-11 09:06:45 on dist-test-slave-cbsf"
server_key: "f3e19bec862a8a5f3ecdf53e564e9bf0"
server_key_iv: "afd309fc56404664e994b44d72dd51ac"
server_key_version: "encryptionkey@0"
I20260811 09:06:45.116039 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "d87a866ab7c048beb25a3ee166c88f56"
format_stamp: "Formatted at 2026-08-11 09:06:45 on dist-test-slave-cbsf"
server_key: "f3e19bec862a8a5f3ecdf53e564e9bf0"
server_key_iv: "afd309fc56404664e994b44d72dd51ac"
server_key_version: "encryptionkey@0"
I20260811 09:06:45.121371 1271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:06:45.124930 1288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:45.126327 1271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:45.126544 1271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "d87a866ab7c048beb25a3ee166c88f56"
format_stamp: "Formatted at 2026-08-11 09:06:45 on dist-test-slave-cbsf"
server_key: "f3e19bec862a8a5f3ecdf53e564e9bf0"
server_key_iv: "afd309fc56404664e994b44d72dd51ac"
server_key_version: "encryptionkey@0"
I20260811 09:06:45.126758 1271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:45.146552 1271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:45.151526 1271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:45.151839 1271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:45.171523 1271 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37087
I20260811 09:06:45.171567 1349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37087 every 8 connection(s)
I20260811 09:06:45.173115 1271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:45.176955 1350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:45.177284 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1271
I20260811 09:06:45.177562 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:45.178098 9377 external_mini_cluster.cc:1428] Setting key d9cbb1c6ac00a07514e7df147c64b1da
I20260811 09:06:45.191951 1350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56: Bootstrap starting.
Aug 11 09:06:45 dist-test-slave-cbsf krb5kdc[1256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:06:45.195399 1350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:45.196849 1350 log.cc:824] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:45.200484 1350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56: No bootstrap required, opened a new log
I20260811 09:06:45.205421 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:45.180403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:40199 (local address 127.0.0.1:37087)
0811 09:06:45.181360 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:45.181376 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:45.181423 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:06:45.184001 (+ 2578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:45.184033 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:45.184593 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:45.185107 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:45.186536 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:45.188651 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:45.189985 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:45.191062 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:45.191237 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:45.195771 (+ 4534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:45.195783 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:45.195810 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:45.195876 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:45.200155 (+ 4279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:45.201196 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:45.201206 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:45.201214 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:45.201441 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:45.202028 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:45.202034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:45.202037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:45.202784 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:45.203155 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:45.203569 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 09:06:45.203947 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":424,"threads_started":1}
I20260811 09:06:45.209609 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:45.210739 1350 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } }
I20260811 09:06:45.211097 1350 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:45.211198 1350 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87a866ab7c048beb25a3ee166c88f56, State: Initialized, Role: FOLLOWER
I20260811 09:06:45.211791 1350 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } }
I20260811 09:06:45.212003 1350 raft_consensus.cc:388] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:45.212209 1350 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:45.212395 1350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:45.215085 1350 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } }
I20260811 09:06:45.215685 1350 leader_election.cc:302] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d87a866ab7c048beb25a3ee166c88f56; no voters:
I20260811 09:06:45.216393 1350 leader_election.cc:288] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:45.216454 1364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:45.218756 1350 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:45.223373 1364 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [term 1 LEADER]: Becoming Leader. State: Replica: d87a866ab7c048beb25a3ee166c88f56, State: Running, Role: LEADER
I20260811 09:06:45.223978 1364 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } }
I20260811 09:06:45.226927 1365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d87a866ab7c048beb25a3ee166c88f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } } }
I20260811 09:06:45.227342 1365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:45.227883 1366 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87a866ab7c048beb25a3ee166c88f56. Latest consensus state: current_term: 1 leader_uuid: "d87a866ab7c048beb25a3ee166c88f56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87a866ab7c048beb25a3ee166c88f56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37087 } } }
I20260811 09:06:45.228582 1366 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:45.234431 1374 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:45.235602 1355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:45.214380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:57793 (local address 127.0.0.1:37087)
0811 09:06:45.214624 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:45.214633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:45.215122 (+ 489us) server_negotiation.cc:407] Connection header received
0811 09:06:45.215328 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:45.215336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:45.215414 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:45.215534 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:45.217709 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:45.218876 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:45.220949 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:45.221878 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:45.222010 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:45.226537 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:45.226547 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:45.226555 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:45.226603 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:45.230956 (+ 4353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:45.232247 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:45.232256 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:45.232262 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:45.232441 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:45.233213 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:45.233220 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:45.233226 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:45.233751 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:45.235060 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:45.235200 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:06:45.235332 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:06:45.241592 1374 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:45.251192 1374 catalog_manager.cc:1347] Generated new cluster ID: 12b61e08c32d4eff96e02af73122bb52
I20260811 09:06:45.251328 1374 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:45.288831 1374 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:45.290520 1374 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:45.304502 1374 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d87a866ab7c048beb25a3ee166c88f56: Generated new TSK 0
I20260811 09:06:45.305529 1374 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:45.321305 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1271
I20260811 09:06:45.322666 1344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:45.425580 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1271
W20260811 09:06:45.449157 1254 connection.cc:535] client connection to 127.0.0.1:37087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37087): Connection reset by peer (error 104)
2026-08-11T09:06:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:45 dist-test-slave-cbsf krb5kdc[1388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:45 dist-test-slave-cbsf krb5kdc[1388](info): set up 2 sockets
Aug 11 09:06:45 dist-test-slave-cbsf krb5kdc[1388](info): commencing operation
krb5kdc: starting...
W20260811 09:06:46.656217 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.006s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:46 dist-test-slave-cbsf krb5kdc[1388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:47.040324 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:47.407737 1405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:47.408144 1405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:47.408341 1405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:47.415784 1405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:47.415994 1405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:47.416118 1405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:47.416218 1405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:47.416319 1405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:47.427273 1405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:47.428764 1405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:47.430154 1405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:47.442503 1410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:47.442536 1411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:47.442696 1414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:47.443519 1405 server_base.cc:1034] running on GCE node
I20260811 09:06:47.444345 1405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:47.445875 1405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:47.447166 1405 hybrid_clock.cc:648] HybridClock initialized: now 1786439207447116 us; error 69 us; skew 500 ppm
I20260811 09:06:47.447558 1405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:47 dist-test-slave-cbsf krb5kdc[1388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439207, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:47.451026 1405 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:47.452764 1405 webserver.cc:460] Webserver started at http://127.0.0.1:42253/ using document root <none> and password file <none>
I20260811 09:06:47.453482 1405 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:47.453644 1405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:47.453959 1405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:47.457213 1405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "cacff7efaba94d6ea4c29a0d0f119e47"
format_stamp: "Formatted at 2026-08-11 09:06:47 on dist-test-slave-cbsf"
server_key: "853e758efc73d059f34215a064772fa9"
server_key_iv: "5f88a4ce5d1ab4343bad91378caf3993"
server_key_version: "encryptionkey@0"
I20260811 09:06:47.458029 1405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "cacff7efaba94d6ea4c29a0d0f119e47"
format_stamp: "Formatted at 2026-08-11 09:06:47 on dist-test-slave-cbsf"
server_key: "853e758efc73d059f34215a064772fa9"
server_key_iv: "5f88a4ce5d1ab4343bad91378caf3993"
server_key_version: "encryptionkey@0"
I20260811 09:06:47.463438 1405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:06:47.467120 1422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:47.468612 1405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:06:47.468842 1405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "cacff7efaba94d6ea4c29a0d0f119e47"
format_stamp: "Formatted at 2026-08-11 09:06:47 on dist-test-slave-cbsf"
server_key: "853e758efc73d059f34215a064772fa9"
server_key_iv: "5f88a4ce5d1ab4343bad91378caf3993"
server_key_version: "encryptionkey@0"
I20260811 09:06:47.469070 1405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:47.481693 1405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:47.486200 1405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:47.486517 1405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:47.507016 1405 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42029
I20260811 09:06:47.507048 1482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42029 every 8 connection(s)
I20260811 09:06:47.508446 1405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:47.512233 1483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:47.518724 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1405
I20260811 09:06:47.518936 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:47.519590 9377 external_mini_cluster.cc:1428] Setting key af145fa4d659fa73d9683f8a4e5d0583
I20260811 09:06:47.523715 1483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47: Bootstrap starting.
I20260811 09:06:47.527413 1483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:47.528774 1483 log.cc:824] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:47.532394 1483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47: No bootstrap required, opened a new log
I20260811 09:06:47.534550 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:47.521570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:39493 (local address 127.0.0.1:42029)
0811 09:06:47.522413 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:47.522425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:47.522467 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:06:47.524315 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:47.524338 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:47.524816 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:47.525145 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:47.525850 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:47.527165 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:47.529210 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:47.530413 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:47.530552 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:47.531900 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:47.531907 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:47.531921 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:47.532568 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:47.532646 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:47.532672 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:47.532869 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:47.533340 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 09:06:47.533571 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":396,"threads_started":1}
I20260811 09:06:47.538671 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:47.541944 1483 raft_consensus.cc:348] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 09:06:47.542580 1483 raft_consensus.cc:374] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:47.542706 1483 raft_consensus.cc:729] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacff7efaba94d6ea4c29a0d0f119e47, State: Initialized, Role: FOLLOWER
I20260811 09:06:47.543488 1483 consensus_queue.cc:260] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 09:06:47.543777 1483 raft_consensus.cc:388] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:47.543929 1483 raft_consensus.cc:482] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:47.544166 1483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:47.546919 1483 raft_consensus.cc:504] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 09:06:47.547585 1483 leader_election.cc:302] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cacff7efaba94d6ea4c29a0d0f119e47; no voters:
I20260811 09:06:47.548280 1483 leader_election.cc:288] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:47.548395 1494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:47.551004 1483 sys_catalog.cc:564] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:47.551333 1494 raft_consensus.cc:686] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [term 1 LEADER]: Becoming Leader. State: Replica: cacff7efaba94d6ea4c29a0d0f119e47, State: Running, Role: LEADER
I20260811 09:06:47.552409 1494 consensus_queue.cc:237] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 09:06:47.552821 1486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:47.544462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:40469 (local address 127.0.0.1:42029)
0811 09:06:47.544651 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:47.544657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:47.545248 (+ 591us) server_negotiation.cc:407] Connection header received
0811 09:06:47.545563 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:47.545568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:47.545645 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:47.545773 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:47.546338 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:47.547110 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:47.549253 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:47.549946 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:47.550151 (+ 205us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:47.551833 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:47.551844 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:47.551852 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:47.551876 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:47.551944 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:47.551965 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:47.552103 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:47.552371 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 09:06:47.552444 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:06:47.556640 1497 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cacff7efaba94d6ea4c29a0d0f119e47. Latest consensus state: current_term: 1 leader_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260811 09:06:47.557245 1497 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:47.557524 1496 sys_catalog.cc:455] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cacff7efaba94d6ea4c29a0d0f119e47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260811 09:06:47.557823 1496 sys_catalog.cc:458] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:47.561246 1508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:47.565889 1508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:47.574730 1508 catalog_manager.cc:1347] Generated new cluster ID: aabbe89531194824b9330278dc5634a5
I20260811 09:06:47.574872 1508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:47.585180 1508 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:47.586525 1508 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:47.593039 1508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cacff7efaba94d6ea4c29a0d0f119e47: Generated new TSK 0
I20260811 09:06:47.593854 1508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:47.604326 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1405
W20260811 09:06:47.605535 1478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.129:39493
W20260811 09:06:47.606360 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1405: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:47.606499 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1405
W20260811 09:06:47.627364 1386 connection.cc:535] client connection to 127.0.0.1:42029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42029): Connection reset by peer (error 104)
2026-08-11T09:06:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2178 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:47 dist-test-slave-cbsf krb5kdc[1518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:47 dist-test-slave-cbsf krb5kdc[1518](info): set up 2 sockets
Aug 11 09:06:47 dist-test-slave-cbsf krb5kdc[1518](info): commencing operation
krb5kdc: starting...
W20260811 09:06:48.829393 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:49 dist-test-slave-cbsf krb5kdc[1518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:49.200966 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:49.563504 1531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:49.563840 1531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:49.564033 1531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:49.571543 1531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:49.571668 1531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:49.571796 1531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:49.571906 1531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:49.571996 1531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:49.582312 1531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:49.583900 1531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:49.585340 1531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:49.597985 1539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:49.598026 1542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:49.597936 1537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:49.599138 1531 server_base.cc:1034] running on GCE node
I20260811 09:06:49.600008 1531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:49.601526 1531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:49.602747 1531 hybrid_clock.cc:648] HybridClock initialized: now 1786439209602684 us; error 61 us; skew 500 ppm
I20260811 09:06:49.603117 1531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:49 dist-test-slave-cbsf krb5kdc[1518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439209, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:49.606683 1531 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:49.608194 1531 webserver.cc:460] Webserver started at http://127.0.0.1:45775/ using document root <none> and password file <none>
I20260811 09:06:49.608969 1531 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:49.609174 1531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:49.609566 1531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:49.613184 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "49236cdfc51247219fcc5e716afef19b"
format_stamp: "Formatted at 2026-08-11 09:06:49 on dist-test-slave-cbsf"
server_key: "2cf0c54aa33f08dad2bc828d7fa4bf73"
server_key_iv: "26ee1f45b20cb4bfdb99d3e756ee97b8"
server_key_version: "encryptionkey@0"
I20260811 09:06:49.613934 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "49236cdfc51247219fcc5e716afef19b"
format_stamp: "Formatted at 2026-08-11 09:06:49 on dist-test-slave-cbsf"
server_key: "2cf0c54aa33f08dad2bc828d7fa4bf73"
server_key_iv: "26ee1f45b20cb4bfdb99d3e756ee97b8"
server_key_version: "encryptionkey@0"
I20260811 09:06:49.619526 1531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:06:49.623181 1550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:49.624572 1531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:06:49.624825 1531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "49236cdfc51247219fcc5e716afef19b"
format_stamp: "Formatted at 2026-08-11 09:06:49 on dist-test-slave-cbsf"
server_key: "2cf0c54aa33f08dad2bc828d7fa4bf73"
server_key_iv: "26ee1f45b20cb4bfdb99d3e756ee97b8"
server_key_version: "encryptionkey@0"
I20260811 09:06:49.625064 1531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:49.630959 1531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:49.635582 1531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:49.635905 1531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:49.655423 1531 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37045
I20260811 09:06:49.655480 1614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37045 every 8 connection(s)
I20260811 09:06:49.656976 1531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:49.660945 1615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:49.667172 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1531
I20260811 09:06:49.667405 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:49.667912 9377 external_mini_cluster.cc:1428] Setting key 06daef60891522f0f896a8a7558e9559
I20260811 09:06:49.672339 1615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b: Bootstrap starting.
I20260811 09:06:49.676013 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:49.669836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:39231 (local address 127.0.0.1:37045)
0811 09:06:49.670743 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:49.670755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:49.670797 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:06:49.672558 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:49.672589 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:49.672985 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:49.673302 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:49.674456 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:49.674463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:49.674475 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:49.674490 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:49.674563 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:49.674584 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:49.674752 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:49.674901 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:06:49.675129 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":441,"threads_started":1}
I20260811 09:06:49.676230 1615 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:49.677634 1615 log.cc:824] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:49.680226 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:49.681133 1615 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b: No bootstrap required, opened a new log
I20260811 09:06:49.688372 1619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:49.685731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:53771 (local address 127.0.0.1:37045)
0811 09:06:49.685946 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:49.685953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:49.686404 (+ 451us) server_negotiation.cc:407] Connection header received
0811 09:06:49.686542 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:49.686548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:49.686617 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:49.686736 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:49.687706 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:49.687713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:49.687718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:49.687737 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:49.687789 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:49.687810 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:49.687946 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:49.687999 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 09:06:49.688062 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:06:49.691491 1615 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } }
I20260811 09:06:49.691896 1615 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:49.692026 1615 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49236cdfc51247219fcc5e716afef19b, State: Initialized, Role: FOLLOWER
I20260811 09:06:49.692742 1615 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } }
I20260811 09:06:49.693024 1615 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:49.693181 1615 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:49.693372 1615 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:49.696163 1615 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } }
I20260811 09:06:49.696779 1615 leader_election.cc:302] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49236cdfc51247219fcc5e716afef19b; no voters:
I20260811 09:06:49.697407 1615 leader_election.cc:288] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:49.697552 1628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:49.698387 1628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [term 1 LEADER]: Becoming Leader. State: Replica: 49236cdfc51247219fcc5e716afef19b, State: Running, Role: LEADER
I20260811 09:06:49.698954 1628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } }
I20260811 09:06:49.699744 1615 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:49.702661 1630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49236cdfc51247219fcc5e716afef19b. Latest consensus state: current_term: 1 leader_uuid: "49236cdfc51247219fcc5e716afef19b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } } }
I20260811 09:06:49.704195 1630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:49.703403 1629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49236cdfc51247219fcc5e716afef19b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49236cdfc51247219fcc5e716afef19b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37045 } } }
I20260811 09:06:49.704841 1629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:49.705524 1637 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:49.710929 1637 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:49.721835 1637 catalog_manager.cc:1347] Generated new cluster ID: d5e7e4247a5a4c80ac7eb72cc59c9f4c
I20260811 09:06:49.721968 1637 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:49.747239 1637 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:49.747964 1637 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:49.768054 1637 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49236cdfc51247219fcc5e716afef19b: Generated new TSK 0
I20260811 09:06:49.769290 1637 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:49.773722 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1531
W20260811 09:06:49.775347 1610 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.129:39231
W20260811 09:06:49.776120 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:49.776293 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1531
2026-08-11T09:06:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2172 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:49 dist-test-slave-cbsf krb5kdc[1651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:49 dist-test-slave-cbsf krb5kdc[1651](info): set up 2 sockets
Aug 11 09:06:49 dist-test-slave-cbsf krb5kdc[1651](info): commencing operation
krb5kdc: starting...
W20260811 09:06:51.016914 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:51 dist-test-slave-cbsf krb5kdc[1651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:51.383838 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:51.741990 1664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:51.742311 1664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:51.742504 1664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:51.750023 1664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:51.750154 1664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:51.750289 1664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:51.750433 1664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:51.750490 1664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:51.761152 1664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38533
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:51.762575 1664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:51.763908 1664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:51.776346 1673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:51.776819 1669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:51.776866 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:51.777040 1664 server_base.cc:1034] running on GCE node
I20260811 09:06:51.778023 1664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:51.779650 1664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:51.780997 1664 hybrid_clock.cc:648] HybridClock initialized: now 1786439211780929 us; error 51 us; skew 500 ppm
I20260811 09:06:51.781399 1664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:51 dist-test-slave-cbsf krb5kdc[1651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439211, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:51.790498 1664 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:51.792222 1664 webserver.cc:460] Webserver started at http://127.0.0.1:45031/ using document root <none> and password file <none>
I20260811 09:06:51.793085 1664 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:51.793248 1664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:51.793586 1664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:51.796861 1664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "6317fa1fb2d24beaaf41fe4505a7ec99"
format_stamp: "Formatted at 2026-08-11 09:06:51 on dist-test-slave-cbsf"
server_key: "721d5ceadce60fb8fe9232577632f239"
server_key_iv: "faa1db12bfb4f9d286cbdfb1bfadeff5"
server_key_version: "encryptionkey@0"
I20260811 09:06:51.797734 1664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6317fa1fb2d24beaaf41fe4505a7ec99"
format_stamp: "Formatted at 2026-08-11 09:06:51 on dist-test-slave-cbsf"
server_key: "721d5ceadce60fb8fe9232577632f239"
server_key_iv: "faa1db12bfb4f9d286cbdfb1bfadeff5"
server_key_version: "encryptionkey@0"
I20260811 09:06:51.803225 1664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:06:51.806804 1681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:51.808085 1664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:06:51.808297 1664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "6317fa1fb2d24beaaf41fe4505a7ec99"
format_stamp: "Formatted at 2026-08-11 09:06:51 on dist-test-slave-cbsf"
server_key: "721d5ceadce60fb8fe9232577632f239"
server_key_iv: "faa1db12bfb4f9d286cbdfb1bfadeff5"
server_key_version: "encryptionkey@0"
I20260811 09:06:51.808519 1664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:51.839747 1664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:51.844167 1664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:51.844506 1664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:51.864784 1664 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38533
I20260811 09:06:51.864847 1742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38533 every 8 connection(s)
I20260811 09:06:51.866263 1664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:51.870196 1744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:51.870194 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1664
I20260811 09:06:51.870401 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:51.870880 9377 external_mini_cluster.cc:1428] Setting key 583776c0f6cc2592d4b8187d5c18d813
I20260811 09:06:51.885367 1744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99: Bootstrap starting.
Aug 11 09:06:51 dist-test-slave-cbsf krb5kdc[1651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:06:51.889565 1744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:51.891016 1744 log.cc:824] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:51.894937 1744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99: No bootstrap required, opened a new log
I20260811 09:06:51.896724 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:51.872805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:46949 (local address 127.0.0.1:38533)
0811 09:06:51.873672 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:51.873684 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:51.873730 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:06:51.875308 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:51.875329 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:51.875738 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:51.876076 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:51.878012 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:51.879281 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:51.881152 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:51.881873 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:51.882004 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:51.887168 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:51.887181 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:51.887203 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:51.887257 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:51.891462 (+ 4205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:51.892635 (+ 1173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:51.892644 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:51.892652 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:51.892883 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:51.893407 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:51.893415 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:51.893421 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:51.894171 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:51.894502 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:51.894944 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 09:06:51.895309 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":403,"threads_started":1}
I20260811 09:06:51.900887 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:51.903932 1744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260811 09:06:51.904331 1744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:51.904465 1744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6317fa1fb2d24beaaf41fe4505a7ec99, State: Initialized, Role: FOLLOWER
I20260811 09:06:51.905139 1744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260811 09:06:51.905462 1744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:51.905663 1744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:51.905866 1744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:51.908618 1744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260811 09:06:51.909256 1744 leader_election.cc:302] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6317fa1fb2d24beaaf41fe4505a7ec99; no voters:
I20260811 09:06:51.910015 1744 leader_election.cc:288] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:51.910120 1755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:51.911321 1755 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [term 1 LEADER]: Becoming Leader. State: Replica: 6317fa1fb2d24beaaf41fe4505a7ec99, State: Running, Role: LEADER
I20260811 09:06:51.912050 1755 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } }
I20260811 09:06:51.912628 1744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:51.915324 1756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } } }
I20260811 09:06:51.915849 1756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:51.918702 1761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:51.918663 1757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6317fa1fb2d24beaaf41fe4505a7ec99. Latest consensus state: current_term: 1 leader_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6317fa1fb2d24beaaf41fe4505a7ec99" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38533 } } }
I20260811 09:06:51.918965 1757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:51.924472 1748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:51.905478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:59115 (local address 127.0.0.1:38533)
0811 09:06:51.905669 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:51.905678 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:51.906225 (+ 547us) server_negotiation.cc:407] Connection header received
0811 09:06:51.906336 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:51.906341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:51.906426 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:51.906538 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:51.908096 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:51.909334 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:51.911023 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:51.911761 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:51.911919 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:51.915359 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:51.915366 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:51.915371 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:51.915410 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:51.919624 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:51.920674 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:51.920681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:51.920687 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:51.920892 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:51.921467 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:51.921473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:51.921479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:51.922051 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:51.922370 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:51.923836 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0811 09:06:51.924052 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 09:06:51.925976 1761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:51.938236 1761 catalog_manager.cc:1347] Generated new cluster ID: 7c42e7b464ab482b9c0b288baf8aa5b1
I20260811 09:06:51.938376 1761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:51.964979 1761 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:51.966367 1761 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:51.975126 1761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6317fa1fb2d24beaaf41fe4505a7ec99: Generated new TSK 0
I20260811 09:06:51.976183 1761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:51.986089 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1664
I20260811 09:06:51.987802 1737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:52.083585 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1664
W20260811 09:06:52.106531 1649 connection.cc:535] client connection to 127.0.0.1:38533 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38533): Connection reset by peer (error 104)
2026-08-11T09:06:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:52 dist-test-slave-cbsf krb5kdc[1781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:52 dist-test-slave-cbsf krb5kdc[1781](info): set up 2 sockets
Aug 11 09:06:52 dist-test-slave-cbsf krb5kdc[1781](info): commencing operation
krb5kdc: starting...
W20260811 09:06:53.322721 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:53 dist-test-slave-cbsf krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:53.688393 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:54.047936 1798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:54.048323 1798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:54.048507 1798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:54.055781 1798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:54.056017 1798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:54.056134 1798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:54.056252 1798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:54.056342 1798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:54.066527 1798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:54.067963 1798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:54.069437 1798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:54.081938 1803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:54.081991 1806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:54.082021 1804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:54.083217 1798 server_base.cc:1034] running on GCE node
I20260811 09:06:54.084045 1798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:54.085687 1798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:54.086941 1798 hybrid_clock.cc:648] HybridClock initialized: now 1786439214086908 us; error 52 us; skew 500 ppm
I20260811 09:06:54.087318 1798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:54 dist-test-slave-cbsf krb5kdc[1781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439214, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:54.090684 1798 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:54.092221 1798 webserver.cc:460] Webserver started at http://127.0.0.1:41743/ using document root <none> and password file <none>
I20260811 09:06:54.092944 1798 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:54.093060 1798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:54.093406 1798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:54.096442 1798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "57d694a30e64462eac656be4ba25166a"
format_stamp: "Formatted at 2026-08-11 09:06:54 on dist-test-slave-cbsf"
server_key: "ae9992969a494530dcfb2c07bef17b5a"
server_key_iv: "6d8fe5d136c0b7b53b8723ec50f78aa0"
server_key_version: "encryptionkey@0"
I20260811 09:06:54.097293 1798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "57d694a30e64462eac656be4ba25166a"
format_stamp: "Formatted at 2026-08-11 09:06:54 on dist-test-slave-cbsf"
server_key: "ae9992969a494530dcfb2c07bef17b5a"
server_key_iv: "6d8fe5d136c0b7b53b8723ec50f78aa0"
server_key_version: "encryptionkey@0"
I20260811 09:06:54.102552 1798 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:06:54.106124 1813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:54.107460 1798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:54.107645 1798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "57d694a30e64462eac656be4ba25166a"
format_stamp: "Formatted at 2026-08-11 09:06:54 on dist-test-slave-cbsf"
server_key: "ae9992969a494530dcfb2c07bef17b5a"
server_key_iv: "6d8fe5d136c0b7b53b8723ec50f78aa0"
server_key_version: "encryptionkey@0"
I20260811 09:06:54.107847 1798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:54.126981 1798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:54.131336 1798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:54.131640 1798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:54.151378 1798 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43931
I20260811 09:06:54.151427 1872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43931 every 8 connection(s)
I20260811 09:06:54.152874 1798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:54.154577 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1798
I20260811 09:06:54.154788 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:54.155267 9377 external_mini_cluster.cc:1428] Setting key 84b3b8bcb0636f1af6d1062d94db5170
I20260811 09:06:54.157429 1873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:54.169528 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:54.157295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:37189 (local address 127.0.0.1:43931)
0811 09:06:54.158186 (+ 891us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:54.158199 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:54.158242 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:06:54.159899 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:54.159921 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:54.160315 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:54.160667 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:54.161307 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:54.162687 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:54.164597 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:54.165472 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:54.165636 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:54.167012 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:54.167019 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:54.167037 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:54.167618 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:54.167700 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:54.167731 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:54.167861 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:54.168224 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 09:06:54.168429 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":445,"threads_started":1}
I20260811 09:06:54.170689 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a: Bootstrap starting.
I20260811 09:06:54.173576 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:54.174276 1873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:54.175555 1873 log.cc:824] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:54.178455 1873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a: No bootstrap required, opened a new log
I20260811 09:06:54.186280 1876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:54.179043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:50171 (local address 127.0.0.1:43931)
0811 09:06:54.179290 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:54.179300 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:54.179733 (+ 433us) server_negotiation.cc:407] Connection header received
0811 09:06:54.179924 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:54.179929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:54.179991 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:54.180081 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:54.180792 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:54.181602 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:54.183568 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:54.184042 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:54.184153 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:54.185455 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:54.185460 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:54.185464 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:54.185481 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:54.185528 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:54.185543 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:54.185626 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:54.185855 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 09:06:54.185900 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:06:54.188601 1873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } }
I20260811 09:06:54.189077 1873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:54.189203 1873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d694a30e64462eac656be4ba25166a, State: Initialized, Role: FOLLOWER
I20260811 09:06:54.189785 1873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } }
I20260811 09:06:54.190088 1873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:54.190244 1873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:54.190428 1873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:54.192636 1873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } }
I20260811 09:06:54.193218 1873 leader_election.cc:302] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57d694a30e64462eac656be4ba25166a; no voters:
I20260811 09:06:54.193823 1873 leader_election.cc:288] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:54.193934 1884 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:54.195336 1884 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [term 1 LEADER]: Becoming Leader. State: Replica: 57d694a30e64462eac656be4ba25166a, State: Running, Role: LEADER
I20260811 09:06:54.195851 1884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } }
I20260811 09:06:54.196195 1873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:54.198865 1885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57d694a30e64462eac656be4ba25166a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } } }
I20260811 09:06:54.199304 1885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:54.199872 1886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57d694a30e64462eac656be4ba25166a. Latest consensus state: current_term: 1 leader_uuid: "57d694a30e64462eac656be4ba25166a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57d694a30e64462eac656be4ba25166a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43931 } } }
I20260811 09:06:54.200161 1886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:54.200295 1896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:54.208604 1896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:54.217994 1896 catalog_manager.cc:1347] Generated new cluster ID: f9532a6ae0cd415d87cd041fd85d2937
I20260811 09:06:54.218120 1896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:54.257740 1896 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:54.259159 1896 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:54.273029 1896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57d694a30e64462eac656be4ba25166a: Generated new TSK 0
I20260811 09:06:54.274160 1896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:54.287026 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1798
W20260811 09:06:54.288370 1867 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.129:37189
W20260811 09:06:54.289587 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1798: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:54.289753 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1798
W20260811 09:06:54.312829 1778 connection.cc:535] client connection to 127.0.0.1:43931 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43931): Connection reset by peer (error 104)
2026-08-11T09:06:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:54 dist-test-slave-cbsf krb5kdc[1909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:54 dist-test-slave-cbsf krb5kdc[1909](info): set up 2 sockets
Aug 11 09:06:54 dist-test-slave-cbsf krb5kdc[1909](info): commencing operation
krb5kdc: starting...
W20260811 09:06:55.523705 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:55 dist-test-slave-cbsf krb5kdc[1909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:06:55.878360 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:56.238739 1924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:56.239051 1924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:56.239250 1924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:56.246690 1924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:56.246810 1924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:56.246920 1924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:56.247030 1924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:56.247100 1924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:56.257432 1924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45031
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:56.258888 1924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:56.260306 1924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:56.273159 1930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:56.273090 1933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:56.273458 1935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:56.274447 1924 server_base.cc:1034] running on GCE node
I20260811 09:06:56.275295 1924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:56.276875 1924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:56.278124 1924 hybrid_clock.cc:648] HybridClock initialized: now 1786439216278074 us; error 39 us; skew 500 ppm
I20260811 09:06:56.278532 1924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:56 dist-test-slave-cbsf krb5kdc[1909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439216, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:56.281912 1924 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:06:56.283582 1924 webserver.cc:460] Webserver started at http://127.0.0.1:46657/ using document root <none> and password file <none>
I20260811 09:06:56.284245 1924 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:56.284349 1924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:56.284627 1924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:56.287787 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "38df5e4bedfd417d9ddc6202b292de96"
format_stamp: "Formatted at 2026-08-11 09:06:56 on dist-test-slave-cbsf"
server_key: "b225c915a47c15cbb293758501e60813"
server_key_iv: "dc2db15e7e1d42a71a97ddbcff2ead0c"
server_key_version: "encryptionkey@0"
I20260811 09:06:56.288528 1924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "38df5e4bedfd417d9ddc6202b292de96"
format_stamp: "Formatted at 2026-08-11 09:06:56 on dist-test-slave-cbsf"
server_key: "b225c915a47c15cbb293758501e60813"
server_key_iv: "dc2db15e7e1d42a71a97ddbcff2ead0c"
server_key_version: "encryptionkey@0"
I20260811 09:06:56.294041 1924 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:06:56.297518 1943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:56.298858 1924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:06:56.299043 1924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "38df5e4bedfd417d9ddc6202b292de96"
format_stamp: "Formatted at 2026-08-11 09:06:56 on dist-test-slave-cbsf"
server_key: "b225c915a47c15cbb293758501e60813"
server_key_iv: "dc2db15e7e1d42a71a97ddbcff2ead0c"
server_key_version: "encryptionkey@0"
I20260811 09:06:56.299247 1924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:56.305490 1924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:56.309876 1924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:56.310199 1924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:56.330849 1924 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45031
I20260811 09:06:56.330905 2004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45031 every 8 connection(s)
I20260811 09:06:56.332401 1924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:56.334651 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 1924
I20260811 09:06:56.334856 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:56.335328 9377 external_mini_cluster.cc:1428] Setting key 980fe33f8e563fe198b95faf2bcc2239
I20260811 09:06:56.336871 2006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:06:56.344470 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:56.337279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:57853 (local address 127.0.0.1:45031)
0811 09:06:56.338111 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:56.338129 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:56.338184 (+ 55us) server_negotiation.cc:407] Connection header received
0811 09:06:56.340359 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:56.340390 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:56.340899 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:56.341358 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:56.342382 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:56.342391 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:56.342409 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:56.342436 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:56.342537 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:56.342570 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:56.342803 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:56.342955 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 09:06:56.343172 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":315,"threads_started":1}
I20260811 09:06:56.348467 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:56.351379 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: Bootstrap starting.
I20260811 09:06:56.355391 2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:56.356390 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:56.353450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:38069 (local address 127.0.0.1:45031)
0811 09:06:56.353724 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:56.353735 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:56.354198 (+ 463us) server_negotiation.cc:407] Connection header received
0811 09:06:56.354440 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:56.354445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:56.354553 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:56.354646 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:56.355671 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:56.355675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:56.355679 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:06:56.355694 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:56.355737 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:06:56.355755 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:56.355852 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:56.356016 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 09:06:56.356072 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:06:56.356885 2006 log.cc:824] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:56.359802 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: No bootstrap required, opened a new log
I20260811 09:06:56.367442 2006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } }
I20260811 09:06:56.367885 2006 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:56.368003 2006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38df5e4bedfd417d9ddc6202b292de96, State: Initialized, Role: FOLLOWER
I20260811 09:06:56.368575 2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } }
I20260811 09:06:56.368880 2006 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:56.369045 2006 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:56.369233 2006 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:56.371731 2006 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } }
I20260811 09:06:56.372336 2006 leader_election.cc:302] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38df5e4bedfd417d9ddc6202b292de96; no voters:
I20260811 09:06:56.373071 2006 leader_election.cc:288] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:56.373232 2016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:56.373961 2016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [term 1 LEADER]: Becoming Leader. State: Replica: 38df5e4bedfd417d9ddc6202b292de96, State: Running, Role: LEADER
I20260811 09:06:56.374491 2016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } }
I20260811 09:06:56.375272 2006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:56.377789 2018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38df5e4bedfd417d9ddc6202b292de96. Latest consensus state: current_term: 1 leader_uuid: "38df5e4bedfd417d9ddc6202b292de96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } } }
I20260811 09:06:56.378314 2018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:56.382831 2017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38df5e4bedfd417d9ddc6202b292de96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38df5e4bedfd417d9ddc6202b292de96" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45031 } } }
I20260811 09:06:56.383199 2017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96 [sys.catalog]: This master's current role is: LEADER
W20260811 09:06:56.390340 2035 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:06:56.390523 2035 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:06:56.390712 2025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:56.392441 2025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:56.400910 2025 catalog_manager.cc:1347] Generated new cluster ID: 885918c0f0084701bf539a5d5fca2fd4
I20260811 09:06:56.401100 2025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:56.417445 2025 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:56.417937 2025 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:56.430014 2025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38df5e4bedfd417d9ddc6202b292de96: Generated new TSK 0
I20260811 09:06:56.430914 2025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:56.440315 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1924
W20260811 09:06:56.441459 1998 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.129:57853
W20260811 09:06:56.442281 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 1924: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:06:56.442435 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 1924
2026-08-11T09:06:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2152 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:56 dist-test-slave-cbsf krb5kdc[2041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:56 dist-test-slave-cbsf krb5kdc[2041](info): set up 2 sockets
Aug 11 09:06:56 dist-test-slave-cbsf krb5kdc[2041](info): commencing operation
krb5kdc: starting...
W20260811 09:06:57.681725 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:06:57 dist-test-slave-cbsf krb5kdc[2041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:06:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:06:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:06:58.042305 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34281
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:34281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:06:58.397523 2056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:06:58.397897 2056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:06:58.398074 2056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:06:58.405471 2056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:06:58.405640 2056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:06:58.405752 2056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:06:58.405874 2056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:06:58.405961 2056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:06:58.416200 2056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:34281
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:34281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:06:58.417861 2056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:06:58.419250 2056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:06:58.431777 2064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:58.431797 2067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:06:58.432313 2065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:06:58.433337 2056 server_base.cc:1034] running on GCE node
I20260811 09:06:58.434142 2056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:06:58.435598 2056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:06:58.436801 2056 hybrid_clock.cc:648] HybridClock initialized: now 1786439218436728 us; error 53 us; skew 500 ppm
I20260811 09:06:58.437162 2056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:06:58 dist-test-slave-cbsf krb5kdc[2041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439218, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:06:58.445070 2056 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:06:58.446691 2056 webserver.cc:460] Webserver started at http://127.9.40.126:37393/ using document root <none> and password file <none>
I20260811 09:06:58.447444 2056 fs_manager.cc:359] Metadata directory not provided
I20260811 09:06:58.447606 2056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:06:58.447942 2056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:06:58.451247 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "9f0ca3bf8e1b4b45aacde67d07d75689"
format_stamp: "Formatted at 2026-08-11 09:06:58 on dist-test-slave-cbsf"
server_key: "978db2c3348027cad8bdec9257be86a6"
server_key_iv: "fad8806fd0015ccc346440447defc4bc"
server_key_version: "encryptionkey@0"
I20260811 09:06:58.452080 2056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "9f0ca3bf8e1b4b45aacde67d07d75689"
format_stamp: "Formatted at 2026-08-11 09:06:58 on dist-test-slave-cbsf"
server_key: "978db2c3348027cad8bdec9257be86a6"
server_key_iv: "fad8806fd0015ccc346440447defc4bc"
server_key_version: "encryptionkey@0"
I20260811 09:06:58.457717 2056 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:06:58.461298 2077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:06:58.462597 2056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:06:58.462795 2056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "9f0ca3bf8e1b4b45aacde67d07d75689"
format_stamp: "Formatted at 2026-08-11 09:06:58 on dist-test-slave-cbsf"
server_key: "978db2c3348027cad8bdec9257be86a6"
server_key_iv: "fad8806fd0015ccc346440447defc4bc"
server_key_version: "encryptionkey@0"
I20260811 09:06:58.463012 2056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:06:58.479156 2056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:06:58.483558 2056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:06:58.483891 2056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:06:58.503870 2056 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:34281
I20260811 09:06:58.503911 2137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:34281 every 8 connection(s)
I20260811 09:06:58.505435 2056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:06:58.508477 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2056
I20260811 09:06:58.508683 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:06:58.509294 9377 external_mini_cluster.cc:1428] Setting key bda798e91eaa0de0f297c6b87d94ac8c
I20260811 09:06:58.509950 2138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:06:58 dist-test-slave-cbsf krb5kdc[2041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:06:58.525239 2138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: Bootstrap starting.
I20260811 09:06:58.529008 2138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: Neither blocks nor log segments found. Creating new log.
I20260811 09:06:58.530149 2138 log.cc:824] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: Log is configured to *not* fsync() on all Append() calls
I20260811 09:06:58.533291 2138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: No bootstrap required, opened a new log
I20260811 09:06:58.534721 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:58.511166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41913 (local address 127.9.40.126:34281)
0811 09:06:58.511994 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:58.512006 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:58.512045 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:06:58.514612 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:58.514634 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:58.515030 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:58.515370 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:58.517336 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:58.518684 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:58.520521 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:58.521297 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:58.521440 (+ 143us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:58.526703 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:58.526712 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:58.526728 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:58.526776 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:58.530093 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:58.531190 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:58.531197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:58.531204 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:58.531384 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:58.531906 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:58.531911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:58.531913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:58.532466 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:58.532889 (+ 423us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:58.533364 (+ 475us) server_negotiation.cc:299] Negotiation successful
0811 09:06:58.533727 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":410,"threads_started":1}
I20260811 09:06:58.539055 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:06:58.542366 2138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } }
I20260811 09:06:58.542785 2138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:06:58.542901 2138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f0ca3bf8e1b4b45aacde67d07d75689, State: Initialized, Role: FOLLOWER
I20260811 09:06:58.543618 2138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } }
I20260811 09:06:58.544003 2138 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:06:58.544169 2138 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:06:58.544380 2138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:06:58.547683 2138 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } }
I20260811 09:06:58.548364 2138 leader_election.cc:302] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f0ca3bf8e1b4b45aacde67d07d75689; no voters:
I20260811 09:06:58.549072 2138 leader_election.cc:288] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:06:58.549183 2150 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:06:58.551605 2138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:06:58.556268 2150 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [term 1 LEADER]: Becoming Leader. State: Replica: 9f0ca3bf8e1b4b45aacde67d07d75689, State: Running, Role: LEADER
I20260811 09:06:58.557216 2150 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } }
I20260811 09:06:58.561024 2151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } } }
I20260811 09:06:58.561513 2151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [sys.catalog]: This master's current role is: LEADER
I20260811 09:06:58.564514 2141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:06:58.543684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48833 (local address 127.9.40.126:34281)
0811 09:06:58.543926 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:06:58.543933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:06:58.544330 (+ 397us) server_negotiation.cc:407] Connection header received
0811 09:06:58.544498 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:06:58.544502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:06:58.544601 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:06:58.544751 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:06:58.546736 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:58.547512 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:58.549323 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:06:58.550337 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:06:58.550510 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:06:58.554283 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:06:58.554293 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:06:58.554300 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:06:58.554357 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:06:58.559097 (+ 4740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:58.560275 (+ 1178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:58.560281 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:58.560284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:58.560429 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:06:58.561121 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:06:58.561126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:06:58.561129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:06:58.561613 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:06:58.561944 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:06:58.563699 (+ 1755us) server_negotiation.cc:299] Negotiation successful
0811 09:06:58.563970 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:06:58.566183 2152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f0ca3bf8e1b4b45aacde67d07d75689. Latest consensus state: current_term: 1 leader_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f0ca3bf8e1b4b45aacde67d07d75689" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 34281 } } }
I20260811 09:06:58.567216 2152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689 [sys.catalog]: This master's current role is: LEADER
W20260811 09:06:58.572862 2168 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:06:58.573006 2168 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:06:58.573163 2170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:06:58.574913 2170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:06:58.583631 2170 catalog_manager.cc:1347] Generated new cluster ID: d418f20eb0d642b79efeb8787af4cc8b
I20260811 09:06:58.583783 2170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:06:58.593698 2170 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:06:58.595537 2170 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:06:58.614023 2170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f0ca3bf8e1b4b45aacde67d07d75689: Generated new TSK 0
I20260811 09:06:58.614959 2170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:06:58.623872 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2056
I20260811 09:06:58.625240 2133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:06:58.721503 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2056
W20260811 09:06:58.745553 2039 connection.cc:535] client connection to 127.9.40.126:34281 recv error: Network error: failed to read from TLS socket (remote: 127.9.40.126:34281): Connection reset by peer (error 104)
2026-08-11T09:06:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:06:58 dist-test-slave-cbsf krb5kdc[2178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:06:58 dist-test-slave-cbsf krb5kdc[2178](info): set up 2 sockets
Aug 11 09:06:58 dist-test-slave-cbsf krb5kdc[2178](info): commencing operation
krb5kdc: starting...
W20260811 09:06:59.952402 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:00 dist-test-slave-cbsf krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:00.312219 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33775
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:00.668985 2191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:00.669307 2191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:00.669494 2191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:00.676615 2191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:00.676764 2191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:00.676855 2191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:00.676997 2191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:00.677085 2191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:00.687129 2191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33775
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:00.688480 2191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:00.689841 2191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:00.702538 2196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:00.702497 2199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:00.702581 2197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:00.703787 2191 server_base.cc:1034] running on GCE node
I20260811 09:07:00.704598 2191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:00.706045 2191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:00.707295 2191 hybrid_clock.cc:648] HybridClock initialized: now 1786439220707251 us; error 57 us; skew 500 ppm
I20260811 09:07:00.707657 2191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:00 dist-test-slave-cbsf krb5kdc[2178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439220, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:00.710862 2191 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:00.712404 2191 webserver.cc:460] Webserver started at http://127.9.40.126:45939/ using document root <none> and password file <none>
I20260811 09:07:00.713197 2191 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:00.713351 2191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:00.713665 2191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:00.716691 2191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "ee2a7eaf3d524661a61234e9a04961aa"
format_stamp: "Formatted at 2026-08-11 09:07:00 on dist-test-slave-cbsf"
server_key: "c491ade4ef064d68b38a39bdf1f96e31"
server_key_iv: "c6f0ee022ac8ee5778dc8c96702890c0"
server_key_version: "encryptionkey@0"
I20260811 09:07:00.717583 2191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "ee2a7eaf3d524661a61234e9a04961aa"
format_stamp: "Formatted at 2026-08-11 09:07:00 on dist-test-slave-cbsf"
server_key: "c491ade4ef064d68b38a39bdf1f96e31"
server_key_iv: "c6f0ee022ac8ee5778dc8c96702890c0"
server_key_version: "encryptionkey@0"
I20260811 09:07:00.722946 2191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:00.726552 2206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:00.727950 2191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:07:00.728173 2191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "ee2a7eaf3d524661a61234e9a04961aa"
format_stamp: "Formatted at 2026-08-11 09:07:00 on dist-test-slave-cbsf"
server_key: "c491ade4ef064d68b38a39bdf1f96e31"
server_key_iv: "c6f0ee022ac8ee5778dc8c96702890c0"
server_key_version: "encryptionkey@0"
I20260811 09:07:00.728384 2191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:00.759050 2191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:00.763566 2191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:00.763891 2191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:00.783494 2191 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33775
I20260811 09:07:00.783532 2269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33775 every 8 connection(s)
I20260811 09:07:00.785017 2191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:00.788666 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2191
I20260811 09:07:00.788931 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:00.788992 2270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:00.789394 9377 external_mini_cluster.cc:1428] Setting key eebb87cec52c674299a01397dbd3441b
I20260811 09:07:00.803689 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:00.791214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38201 (local address 127.9.40.126:33775)
0811 09:07:00.792176 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:00.792188 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:00.792231 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:07:00.793967 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:00.793991 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:00.794410 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:00.794762 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:00.795422 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:00.796885 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:00.798846 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:00.799584 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:00.799735 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:00.801217 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:00.801225 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:00.801241 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:00.801854 (+ 613us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:00.801945 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:00.801974 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:00.802108 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:00.802436 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 09:07:00.802636 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":504,"threads_started":1}
I20260811 09:07:00.803826 2270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa: Bootstrap starting.
I20260811 09:07:00.807693 2270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:00.807890 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:00.808893 2270 log.cc:824] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:00.812062 2270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa: No bootstrap required, opened a new log
I20260811 09:07:00.819487 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:00.812908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47255 (local address 127.9.40.126:33775)
0811 09:07:00.813129 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:00.813137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:00.813610 (+ 473us) server_negotiation.cc:407] Connection header received
0811 09:07:00.813841 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:00.813847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:00.813920 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:00.814043 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:00.814566 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:00.815394 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:00.816758 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:00.817297 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:00.817459 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:00.818702 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:00.818707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:00.818714 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:00.818734 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:00.818772 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:00.818787 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:00.818932 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:00.819135 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 09:07:00.819197 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:07:00.820735 2270 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } }
I20260811 09:07:00.821234 2270 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:00.821341 2270 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee2a7eaf3d524661a61234e9a04961aa, State: Initialized, Role: FOLLOWER
I20260811 09:07:00.821913 2270 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } }
I20260811 09:07:00.822135 2270 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:00.822239 2270 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:00.822407 2270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:00.825212 2270 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } }
I20260811 09:07:00.825695 2270 leader_election.cc:302] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee2a7eaf3d524661a61234e9a04961aa; no voters:
I20260811 09:07:00.826248 2270 leader_election.cc:288] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:00.826408 2281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:00.827139 2281 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [term 1 LEADER]: Becoming Leader. State: Replica: ee2a7eaf3d524661a61234e9a04961aa, State: Running, Role: LEADER
I20260811 09:07:00.827751 2281 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } }
I20260811 09:07:00.828488 2270 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:00.830729 2282 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee2a7eaf3d524661a61234e9a04961aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } } }
I20260811 09:07:00.831131 2282 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:00.832448 2283 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee2a7eaf3d524661a61234e9a04961aa. Latest consensus state: current_term: 1 leader_uuid: "ee2a7eaf3d524661a61234e9a04961aa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee2a7eaf3d524661a61234e9a04961aa" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33775 } } }
I20260811 09:07:00.832885 2283 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:00.835088 2287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:00.840420 2287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:00.850425 2287 catalog_manager.cc:1347] Generated new cluster ID: 1d1de69da1e343ea86cafb97aa37d10f
I20260811 09:07:00.850564 2287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:00.868865 2287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:00.870857 2287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:00.888861 2287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee2a7eaf3d524661a61234e9a04961aa: Generated new TSK 0
I20260811 09:07:00.890027 2287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:00.901332 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2191
W20260811 09:07:00.902896 2265 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38201
W20260811 09:07:00.903841 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2191: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:00.904017 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2191
W20260811 09:07:00.927552 2175 connection.cc:535] client connection to 127.9.40.126:33775 recv error: Network error: failed to read from TLS socket (remote: 127.9.40.126:33775): Connection reset by peer (error 104)
2026-08-11T09:07:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2181 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:01 dist-test-slave-cbsf krb5kdc[2305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:01 dist-test-slave-cbsf krb5kdc[2305](info): set up 2 sockets
Aug 11 09:07:01 dist-test-slave-cbsf krb5kdc[2305](info): commencing operation
krb5kdc: starting...
W20260811 09:07:02.137445 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:02 dist-test-slave-cbsf krb5kdc[2305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:02.497906 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35587
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:02.854336 2319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:02.854708 2319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:02.854882 2319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:02.862264 2319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:02.862445 2319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:02.862566 2319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:02.862682 2319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:02.862771 2319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:02.873005 2319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35587
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:02.874505 2319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:02.875936 2319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:02.889134 2329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:02.889336 2327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:02.889353 2326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:02.894515 2319 server_base.cc:1034] running on GCE node
I20260811 09:07:02.895469 2319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:02.897260 2319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:02.898524 2319 hybrid_clock.cc:648] HybridClock initialized: now 1786439222898484 us; error 56 us; skew 500 ppm
I20260811 09:07:02.898919 2319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:02 dist-test-slave-cbsf krb5kdc[2305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439222, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:02.902397 2319 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:02.904052 2319 webserver.cc:460] Webserver started at http://127.9.40.126:33821/ using document root <none> and password file <none>
I20260811 09:07:02.904886 2319 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:02.905041 2319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:02.905373 2319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:02.908576 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "c69eb5def8474d1caf90644017c71deb"
format_stamp: "Formatted at 2026-08-11 09:07:02 on dist-test-slave-cbsf"
server_key: "9140ed73f84a6e7445cb27338158dc50"
server_key_iv: "8452012617a6d0a2827553ea4024d505"
server_key_version: "encryptionkey@0"
I20260811 09:07:02.909454 2319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "c69eb5def8474d1caf90644017c71deb"
format_stamp: "Formatted at 2026-08-11 09:07:02 on dist-test-slave-cbsf"
server_key: "9140ed73f84a6e7445cb27338158dc50"
server_key_iv: "8452012617a6d0a2827553ea4024d505"
server_key_version: "encryptionkey@0"
I20260811 09:07:02.914866 2319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:02.918637 2336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:02.919963 2319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:07:02.920156 2319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "c69eb5def8474d1caf90644017c71deb"
format_stamp: "Formatted at 2026-08-11 09:07:02 on dist-test-slave-cbsf"
server_key: "9140ed73f84a6e7445cb27338158dc50"
server_key_iv: "8452012617a6d0a2827553ea4024d505"
server_key_version: "encryptionkey@0"
I20260811 09:07:02.920367 2319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:02.927192 2319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:02.931315 2319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:02.931633 2319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:02.951390 2319 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35587
I20260811 09:07:02.951493 2398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35587 every 8 connection(s)
I20260811 09:07:02.953265 2319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:02.957216 2399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:02.963502 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2319
I20260811 09:07:02.963722 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:02.964190 9377 external_mini_cluster.cc:1428] Setting key bb6ac759d260445e6fe10d19ab72f67a
I20260811 09:07:02.970342 2399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: Bootstrap starting.
I20260811 09:07:02.974517 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:02.966259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45057 (local address 127.9.40.126:35587)
0811 09:07:02.967339 (+ 1080us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:02.967356 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:02.967416 (+ 60us) server_negotiation.cc:407] Connection header received
0811 09:07:02.970112 (+ 2696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:02.970145 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:02.970628 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:02.971155 (+ 527us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:02.972287 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:02.972298 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:02.972324 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:02.972348 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:02.972438 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:02.972475 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:02.972774 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:02.972940 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:07:02.973159 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":454,"threads_started":1}
I20260811 09:07:02.975235 2399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:02.976472 2399 log.cc:824] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:02.978610 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:02.979913 2399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: No bootstrap required, opened a new log
I20260811 09:07:02.986874 2403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:02.983293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55421 (local address 127.9.40.126:35587)
0811 09:07:02.984636 (+ 1343us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:02.984641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:02.984657 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:07:02.984843 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:02.984848 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:02.984905 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:02.985006 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:02.986082 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:02.986087 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:02.986090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:02.986124 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:02.986166 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:02.986181 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:02.986337 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:02.986515 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 09:07:02.986585 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127}
I20260811 09:07:02.990155 2399 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } }
I20260811 09:07:02.990623 2399 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:02.990775 2399 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69eb5def8474d1caf90644017c71deb, State: Initialized, Role: FOLLOWER
I20260811 09:07:02.991564 2399 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } }
I20260811 09:07:02.991905 2399 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:02.992110 2399 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:02.992341 2399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:02.996644 2399 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } }
I20260811 09:07:02.997314 2399 leader_election.cc:302] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69eb5def8474d1caf90644017c71deb; no voters:
I20260811 09:07:02.998155 2399 leader_election.cc:288] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:02.998257 2413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:03.000801 2399 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:03.001053 2413 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [term 1 LEADER]: Becoming Leader. State: Replica: c69eb5def8474d1caf90644017c71deb, State: Running, Role: LEADER
I20260811 09:07:03.001582 2413 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } }
I20260811 09:07:03.004179 2414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69eb5def8474d1caf90644017c71deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } } }
I20260811 09:07:03.004727 2414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:03.005206 2415 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69eb5def8474d1caf90644017c71deb. Latest consensus state: current_term: 1 leader_uuid: "c69eb5def8474d1caf90644017c71deb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69eb5def8474d1caf90644017c71deb" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35587 } } }
I20260811 09:07:03.005563 2415 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb [sys.catalog]: This master's current role is: LEADER
W20260811 09:07:03.017822 2429 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:07:03.017975 2429 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:07:03.018157 2420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:03.019884 2420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:03.028656 2420 catalog_manager.cc:1347] Generated new cluster ID: afec57e7a18c4d9d9f714033e7b66788
I20260811 09:07:03.028811 2420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:03.048504 2420 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:03.048992 2420 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:03.060599 2420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69eb5def8474d1caf90644017c71deb: Generated new TSK 0
I20260811 09:07:03.061538 2420 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:03.068725 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2319
W20260811 09:07:03.070271 2392 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45057
W20260811 09:07:03.070998 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2319: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:03.071163 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2319
2026-08-11T09:07:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2168 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:03 dist-test-slave-cbsf krb5kdc[2435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:03 dist-test-slave-cbsf krb5kdc[2435](info): set up 2 sockets
Aug 11 09:07:03 dist-test-slave-cbsf krb5kdc[2435](info): commencing operation
krb5kdc: starting...
W20260811 09:07:04.287245 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:04 dist-test-slave-cbsf krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:04.654448 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46715
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:05.035022 2450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:05.035362 2450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:05.035490 2450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:05.043149 2450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:05.043303 2450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:05.043368 2450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:05.043449 2450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:05.043502 2450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:05.054404 2450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:05.055773 2450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:05.057122 2450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:05.069691 2456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:05.069731 2459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:05.069788 2457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:05.070907 2450 server_base.cc:1034] running on GCE node
I20260811 09:07:05.071714 2450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:05.073334 2450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:05.074580 2450 hybrid_clock.cc:648] HybridClock initialized: now 1786439225074533 us; error 37 us; skew 500 ppm
I20260811 09:07:05.075006 2450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:05 dist-test-slave-cbsf krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439225, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:05.083606 2450 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:05.085412 2450 webserver.cc:460] Webserver started at http://127.9.40.126:38179/ using document root <none> and password file <none>
I20260811 09:07:05.086134 2450 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:05.086251 2450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:05.086529 2450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:05.089807 2450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "535438f278d24503822af71b70699a71"
format_stamp: "Formatted at 2026-08-11 09:07:05 on dist-test-slave-cbsf"
server_key: "6edb6ba69388993a46509d06a463c551"
server_key_iv: "e65266321f45fa86d50b28faf96afc0e"
server_key_version: "encryptionkey@0"
I20260811 09:07:05.090570 2450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "535438f278d24503822af71b70699a71"
format_stamp: "Formatted at 2026-08-11 09:07:05 on dist-test-slave-cbsf"
server_key: "6edb6ba69388993a46509d06a463c551"
server_key_iv: "e65266321f45fa86d50b28faf96afc0e"
server_key_version: "encryptionkey@0"
I20260811 09:07:05.096058 2450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:07:05.099749 2468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:05.101187 2450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:05.101382 2450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "535438f278d24503822af71b70699a71"
format_stamp: "Formatted at 2026-08-11 09:07:05 on dist-test-slave-cbsf"
server_key: "6edb6ba69388993a46509d06a463c551"
server_key_iv: "e65266321f45fa86d50b28faf96afc0e"
server_key_version: "encryptionkey@0"
I20260811 09:07:05.101593 2450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:05.126981 2450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:05.131819 2450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:05.132164 2450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:05.152424 2450 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46715
I20260811 09:07:05.152527 2527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46715 every 8 connection(s)
I20260811 09:07:05.153964 2450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:05.157858 2528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:05.163028 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2450
I20260811 09:07:05.163239 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:05.163836 9377 external_mini_cluster.cc:1428] Setting key 44f1418cb9a2b3106c7ab72c8e49ef7b
I20260811 09:07:05.169798 2528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: Bootstrap starting.
I20260811 09:07:05.173684 2528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:05.174961 2528 log.cc:824] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:05.178697 2528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: No bootstrap required, opened a new log
Aug 11 09:07:05 dist-test-slave-cbsf krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:05.190140 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:05.165815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55015 (local address 127.9.40.126:46715)
0811 09:07:05.166708 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:05.166721 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:05.166762 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:07:05.168617 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:05.168645 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:05.169126 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:05.169458 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:05.171351 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:05.172792 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:05.174589 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:05.175355 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:05.175567 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:05.180555 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:05.180566 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:05.180592 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:05.180648 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:05.185037 (+ 4389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:05.185879 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:05.185889 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:05.185899 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:05.186130 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:05.186521 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:05.186532 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:05.186541 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:05.187367 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:05.187698 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:05.188084 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 09:07:05.188629 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":418,"threads_started":1}
I20260811 09:07:05.190006 2528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } }
I20260811 09:07:05.190604 2528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:05.190766 2528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 535438f278d24503822af71b70699a71, State: Initialized, Role: FOLLOWER
I20260811 09:07:05.191349 2528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } }
I20260811 09:07:05.191707 2528 raft_consensus.cc:388] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:05.191910 2528 raft_consensus.cc:482] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:05.192106 2528 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:05.194401 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:05.194967 2528 raft_consensus.cc:504] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } }
I20260811 09:07:05.195499 2528 leader_election.cc:302] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 535438f278d24503822af71b70699a71; no voters:
I20260811 09:07:05.196113 2528 leader_election.cc:288] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:05.196277 2533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:05.198283 2528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:05.203897 2533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [term 1 LEADER]: Becoming Leader. State: Replica: 535438f278d24503822af71b70699a71, State: Running, Role: LEADER
I20260811 09:07:05.204795 2533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } }
I20260811 09:07:05.208320 2534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "535438f278d24503822af71b70699a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } } }
I20260811 09:07:05.208765 2534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:05.209482 2535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 535438f278d24503822af71b70699a71. Latest consensus state: current_term: 1 leader_uuid: "535438f278d24503822af71b70699a71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "535438f278d24503822af71b70699a71" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46715 } } }
I20260811 09:07:05.209815 2535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:05.227866 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:05.205895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55865 (local address 127.9.40.126:46715)
0811 09:07:05.206100 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:05.206107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:05.206796 (+ 689us) server_negotiation.cc:407] Connection header received
0811 09:07:05.206997 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:05.207002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:05.207090 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:05.207233 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:05.209126 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:05.210553 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:05.213182 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:05.214221 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:05.214378 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:05.218889 (+ 4511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:05.218899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:05.218907 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:05.218973 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:05.224171 (+ 5198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:05.225350 (+ 1179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:05.225358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:05.225364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:05.225547 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:05.226060 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:05.226065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:05.226068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:05.226534 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:05.226799 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:05.227137 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 09:07:05.227391 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:07:05.230335 2556 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:07:05.230484 2556 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:07:05.230647 2554 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:05.232961 2554 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:05.242643 2554 catalog_manager.cc:1347] Generated new cluster ID: 9c5466279de34d0d93647452edfbaa9a
I20260811 09:07:05.242794 2554 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:05.257045 2554 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:05.258965 2554 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:05.273219 2554 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 535438f278d24503822af71b70699a71: Generated new TSK 0
I20260811 09:07:05.274410 2554 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:05.287047 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2450
I20260811 09:07:05.288573 2522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:05.384100 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2450
2026-08-11T09:07:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:05 dist-test-slave-cbsf krb5kdc[2563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:05 dist-test-slave-cbsf krb5kdc[2563](info): set up 2 sockets
Aug 11 09:07:05 dist-test-slave-cbsf krb5kdc[2563](info): commencing operation
krb5kdc: starting...
W20260811 09:07:06.617396 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.002s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:06 dist-test-slave-cbsf krb5kdc[2563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439226, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:06.993849 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42377
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:42377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:07.351294 2580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:07.351631 2580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:07.351729 2580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:07.359068 2580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:07.359186 2580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:07.359234 2580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:07.359297 2580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:07.359388 2580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:07.369822 2580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:42377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:07.371210 2580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:07.372601 2580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:07.385080 2586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:07.384945 2587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:07.385051 2589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:07.386008 2580 server_base.cc:1034] running on GCE node
I20260811 09:07:07.386837 2580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:07.388320 2580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:07.389530 2580 hybrid_clock.cc:648] HybridClock initialized: now 1786439227389486 us; error 34 us; skew 500 ppm
I20260811 09:07:07.389911 2580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:07 dist-test-slave-cbsf krb5kdc[2563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439227, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:07.393342 2580 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:07.394994 2580 webserver.cc:460] Webserver started at http://127.9.40.126:41263/ using document root <none> and password file <none>
I20260811 09:07:07.395767 2580 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:07.395929 2580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:07.396293 2580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:07.399799 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "228403e6f9cb4a10948817ef8155a1a4"
format_stamp: "Formatted at 2026-08-11 09:07:07 on dist-test-slave-cbsf"
server_key: "772819bccedea27e14b151abf7098a42"
server_key_iv: "6095a9c3f56f1553c47b24383dd0a1aa"
server_key_version: "encryptionkey@0"
I20260811 09:07:07.400612 2580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "228403e6f9cb4a10948817ef8155a1a4"
format_stamp: "Formatted at 2026-08-11 09:07:07 on dist-test-slave-cbsf"
server_key: "772819bccedea27e14b151abf7098a42"
server_key_iv: "6095a9c3f56f1553c47b24383dd0a1aa"
server_key_version: "encryptionkey@0"
I20260811 09:07:07.406004 2580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:07:07.409554 2596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:07.410897 2580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:07:07.411103 2580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "228403e6f9cb4a10948817ef8155a1a4"
format_stamp: "Formatted at 2026-08-11 09:07:07 on dist-test-slave-cbsf"
server_key: "772819bccedea27e14b151abf7098a42"
server_key_iv: "6095a9c3f56f1553c47b24383dd0a1aa"
server_key_version: "encryptionkey@0"
I20260811 09:07:07.411325 2580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:07.431387 2580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:07.436290 2580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:07.436616 2580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:07.456326 2580 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:42377
I20260811 09:07:07.456357 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:42377 every 8 connection(s)
I20260811 09:07:07.457844 2580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:07.460887 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2580
I20260811 09:07:07.461081 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:07.461642 9377 external_mini_cluster.cc:1428] Setting key 5d023396e4f488543e9b7b81dd23a068
I20260811 09:07:07.462280 2657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:07.474679 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:07.463538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41981 (local address 127.9.40.126:42377)
0811 09:07:07.464328 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:07.464341 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:07.464380 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:07:07.466001 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:07.466023 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:07.466415 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:07.466867 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:07.467430 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:07.468848 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:07.470177 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:07.470857 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:07.471065 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:07.472358 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:07.472368 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:07.472382 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:07.473011 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:07.473088 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:07.473112 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:07.473252 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:07.473468 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 09:07:07.473732 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":340,"threads_started":1}
I20260811 09:07:07.474736 2657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4: Bootstrap starting.
I20260811 09:07:07.478466 2657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:07.478471 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:07.479468 2657 log.cc:824] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:07.482781 2657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4: No bootstrap required, opened a new log
I20260811 09:07:07.489339 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:07.483030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36523 (local address 127.9.40.126:42377)
0811 09:07:07.483230 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:07.483238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:07.483596 (+ 358us) server_negotiation.cc:407] Connection header received
0811 09:07:07.483882 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:07.483886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:07.483947 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:07.484054 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:07.484597 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:07.485441 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:07.486770 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:07.487257 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:07.487422 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:07.488533 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:07.488538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:07.488541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:07.488559 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:07.488595 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:07.488610 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:07.488735 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:07.488905 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 09:07:07.489004 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:07:07.491175 2657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } }
I20260811 09:07:07.491591 2657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:07.491714 2657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 228403e6f9cb4a10948817ef8155a1a4, State: Initialized, Role: FOLLOWER
I20260811 09:07:07.492270 2657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } }
I20260811 09:07:07.492525 2657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:07.492681 2657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:07.492920 2657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:07.495288 2657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } }
I20260811 09:07:07.495831 2657 leader_election.cc:302] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 228403e6f9cb4a10948817ef8155a1a4; no voters:
I20260811 09:07:07.496426 2657 leader_election.cc:288] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:07.496565 2669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:07.497421 2669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [term 1 LEADER]: Becoming Leader. State: Replica: 228403e6f9cb4a10948817ef8155a1a4, State: Running, Role: LEADER
I20260811 09:07:07.498001 2669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } }
I20260811 09:07:07.498636 2657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:07.501410 2671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 228403e6f9cb4a10948817ef8155a1a4. Latest consensus state: current_term: 1 leader_uuid: "228403e6f9cb4a10948817ef8155a1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } } }
I20260811 09:07:07.501824 2671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:07.502720 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "228403e6f9cb4a10948817ef8155a1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "228403e6f9cb4a10948817ef8155a1a4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42377 } } }
I20260811 09:07:07.503072 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:07.504205 2679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:07.508960 2679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:07.517633 2679 catalog_manager.cc:1347] Generated new cluster ID: 9216b7ecc4e749dcb2e748256735d53b
I20260811 09:07:07.517755 2679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:07.538074 2679 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:07.540104 2679 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:07.555924 2679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 228403e6f9cb4a10948817ef8155a1a4: Generated new TSK 0
I20260811 09:07:07.557106 2679 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:07.567365 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2580
W20260811 09:07:07.568564 2650 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41981
W20260811 09:07:07.569348 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2580: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:07.569512 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2580
2026-08-11T09:07:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2182 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:07 dist-test-slave-cbsf krb5kdc[2692](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:07 dist-test-slave-cbsf krb5kdc[2692](info): set up 2 sockets
Aug 11 09:07:07 dist-test-slave-cbsf krb5kdc[2692](info): commencing operation
krb5kdc: starting...
W20260811 09:07:08.802287 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:09 dist-test-slave-cbsf krb5kdc[2692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439229, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:09.174283 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33377
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:09.532419 2706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:09.532855 2706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:09.533035 2706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:09.540512 2706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:09.540642 2706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:09.540750 2706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:09.540864 2706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:09.540927 2706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:09.551290 2706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:09.552779 2706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:09.554162 2706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:09.566699 2712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:09.566658 2713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:09.566751 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:09.567407 2706 server_base.cc:1034] running on GCE node
I20260811 09:07:09.568425 2706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:09.570055 2706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:09.571302 2706 hybrid_clock.cc:648] HybridClock initialized: now 1786439229571261 us; error 37 us; skew 500 ppm
I20260811 09:07:09.571689 2706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:09 dist-test-slave-cbsf krb5kdc[2692](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439229, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:09.574996 2706 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:09.576680 2706 webserver.cc:460] Webserver started at http://127.9.40.126:38293/ using document root <none> and password file <none>
I20260811 09:07:09.577504 2706 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:09.577670 2706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:09.578012 2706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:09.581328 2706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "138732a848874589855758b328247a22"
format_stamp: "Formatted at 2026-08-11 09:07:09 on dist-test-slave-cbsf"
server_key: "e3ea8e1214cf2a8758d82b8db1ea2c77"
server_key_iv: "2baa4f3e8198b30ecf29ded223a4b303"
server_key_version: "encryptionkey@0"
I20260811 09:07:09.582173 2706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "138732a848874589855758b328247a22"
format_stamp: "Formatted at 2026-08-11 09:07:09 on dist-test-slave-cbsf"
server_key: "e3ea8e1214cf2a8758d82b8db1ea2c77"
server_key_iv: "2baa4f3e8198b30ecf29ded223a4b303"
server_key_version: "encryptionkey@0"
I20260811 09:07:09.587877 2706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:07:09.591735 2724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:09.593169 2706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:09.593366 2706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "138732a848874589855758b328247a22"
format_stamp: "Formatted at 2026-08-11 09:07:09 on dist-test-slave-cbsf"
server_key: "e3ea8e1214cf2a8758d82b8db1ea2c77"
server_key_iv: "2baa4f3e8198b30ecf29ded223a4b303"
server_key_version: "encryptionkey@0"
I20260811 09:07:09.593588 2706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:09.599666 2706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:09.603905 2706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:09.604218 2706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:09.624090 2706 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33377
I20260811 09:07:09.624140 2785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33377 every 8 connection(s)
I20260811 09:07:09.625705 2706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:09.629557 2786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:09.631953 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2706
I20260811 09:07:09.632164 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:09.632735 9377 external_mini_cluster.cc:1428] Setting key c9c0a4383ee500ad72f201a79bc0065d
I20260811 09:07:09.641005 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:09.634645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41289 (local address 127.9.40.126:33377)
0811 09:07:09.635551 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:09.635563 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:09.635607 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:07:09.637437 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:09.637462 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:09.637809 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:09.638144 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:09.639277 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:09.639284 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:09.639297 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:09.639311 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:09.639383 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:09.639406 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:09.639586 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:09.639694 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 09:07:09.639917 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":429,"threads_started":1}
I20260811 09:07:09.642506 2786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22: Bootstrap starting.
I20260811 09:07:09.644829 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:09.646059 2786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:09.647360 2786 log.cc:824] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:09.650554 2786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22: No bootstrap required, opened a new log
I20260811 09:07:09.652729 2790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:09.650243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48377 (local address 127.9.40.126:33377)
0811 09:07:09.650419 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:09.650425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:09.650832 (+ 407us) server_negotiation.cc:407] Connection header received
0811 09:07:09.650971 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:09.650976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:09.651047 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:09.651175 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:09.652045 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:09.652051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:09.652055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:07:09.652073 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:09.652121 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:07:09.652143 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:09.652269 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:09.652320 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 09:07:09.652381 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:07:09.659773 2786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } }
I20260811 09:07:09.660252 2786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:09.660342 2786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138732a848874589855758b328247a22, State: Initialized, Role: FOLLOWER
I20260811 09:07:09.660964 2786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } }
I20260811 09:07:09.661186 2786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:09.661301 2786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:09.661428 2786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:09.663676 2786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } }
I20260811 09:07:09.664152 2786 leader_election.cc:302] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 138732a848874589855758b328247a22; no voters:
I20260811 09:07:09.664836 2786 leader_election.cc:288] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:09.665010 2797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:09.665729 2797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [term 1 LEADER]: Becoming Leader. State: Replica: 138732a848874589855758b328247a22, State: Running, Role: LEADER
I20260811 09:07:09.666280 2797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } }
I20260811 09:07:09.667321 2786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:09.669243 2799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 138732a848874589855758b328247a22. Latest consensus state: current_term: 1 leader_uuid: "138732a848874589855758b328247a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } } }
I20260811 09:07:09.669697 2799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:09.670485 2798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "138732a848874589855758b328247a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "138732a848874589855758b328247a22" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33377 } } }
I20260811 09:07:09.670888 2798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:09.671838 2807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:09.677440 2807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:09.687147 2807 catalog_manager.cc:1347] Generated new cluster ID: d251420c649d453d9cc54705f6affe01
I20260811 09:07:09.687284 2807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:09.711512 2807 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:09.712066 2807 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:09.726833 2807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 138732a848874589855758b328247a22: Generated new TSK 0
I20260811 09:07:09.728052 2807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:09.744298 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2706
W20260811 09:07:09.745505 2780 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:41289
W20260811 09:07:09.746264 9377 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2706: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:09.746425 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2706
2026-08-11T09:07:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2175 ms)
[----------] 18 tests from AuthTokenIssuingTest (39862 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:09 dist-test-slave-cbsf krb5kdc[2821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:09 dist-test-slave-cbsf krb5kdc[2821](info): set up 2 sockets
Aug 11 09:07:09 dist-test-slave-cbsf krb5kdc[2821](info): commencing operation
krb5kdc: starting...
W20260811 09:07:10.974486 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.005s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:11 dist-test-slave-cbsf krb5kdc[2821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:11.348069 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44941
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:11.706821 2834 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:11.707196 2834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:11.707329 2834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:11.707470 2834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:11.715085 2834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:11.715174 2834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:11.715233 2834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:11.715278 2834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:11.715392 2834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:11.725829 2834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:11.727242 2834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:11.728579 2834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:11.741138 2839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:11.741068 2840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:11.741034 2842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:11.742240 2834 server_base.cc:1034] running on GCE node
I20260811 09:07:11.743114 2834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:11.744547 2834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:11.745813 2834 hybrid_clock.cc:648] HybridClock initialized: now 1786439231745758 us; error 39 us; skew 500 ppm
I20260811 09:07:11.746172 2834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:11 dist-test-slave-cbsf krb5kdc[2821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:11.752962 2834 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:11.754604 2834 webserver.cc:460] Webserver started at http://127.9.40.126:38973/ using document root <none> and password file <none>
I20260811 09:07:11.755343 2834 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:11.755479 2834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:11.755812 2834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:11.759279 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "6fe2b58f65d84c59a491b3a6d8cea674"
format_stamp: "Formatted at 2026-08-11 09:07:11 on dist-test-slave-cbsf"
server_key: "49e1bd3ca3449a912bed2d1758c51948"
server_key_iv: "fb4aa016c021cca2e55f9cfa2cfb4b1e"
server_key_version: "encryptionkey@0"
I20260811 09:07:11.760106 2834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6fe2b58f65d84c59a491b3a6d8cea674"
format_stamp: "Formatted at 2026-08-11 09:07:11 on dist-test-slave-cbsf"
server_key: "49e1bd3ca3449a912bed2d1758c51948"
server_key_iv: "fb4aa016c021cca2e55f9cfa2cfb4b1e"
server_key_version: "encryptionkey@0"
I20260811 09:07:11.765583 2834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:07:11.769080 2849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:11.770385 2834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:07:11.770589 2834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "6fe2b58f65d84c59a491b3a6d8cea674"
format_stamp: "Formatted at 2026-08-11 09:07:11 on dist-test-slave-cbsf"
server_key: "49e1bd3ca3449a912bed2d1758c51948"
server_key_iv: "fb4aa016c021cca2e55f9cfa2cfb4b1e"
server_key_version: "encryptionkey@0"
I20260811 09:07:11.770836 2834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:11.788563 2834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:11.792721 2834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:11.793020 2834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:11.814291 2834 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44941
I20260811 09:07:11.814311 2911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44941 every 8 connection(s)
I20260811 09:07:11.815750 2834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:11.819475 2913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:11.825215 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2834
I20260811 09:07:11.825456 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:11.826097 9377 external_mini_cluster.cc:1428] Setting key 63cb9716896eb0bb01c7073d72ef3362
I20260811 09:07:11.826929 2913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:11 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:11.853679 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.830151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44941 (local address 127.0.0.1:47148)
0811 09:07:11.831379 (+ 1228us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.831439 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:11.834218 (+ 2779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:11.835065 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:11.835074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:11.835920 (+ 846us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:11.836448 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:11.836479 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:11.838148 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:11.838156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:11.840035 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:11.840053 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:11.841142 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:11.841152 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:11.841399 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:11.842166 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:11.842188 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:11.844649 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:11.849449 (+ 4800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:11.849460 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:11.849470 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:11.849890 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:11.850432 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:11.850435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:11.850437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:11.850598 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:11.851624 (+ 1026us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:11.851631 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:11.851849 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 09:07:11.852066 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 09:07:11.852489 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"spinlock_wait_cycles":40320,"thread_start_us":743,"threads_started":1}
I20260811 09:07:11.853724 2916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.830153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47148 (local address 127.9.40.126:44941)
0811 09:07:11.831089 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:11.831104 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:11.831581 (+ 477us) server_negotiation.cc:407] Connection header received
0811 09:07:11.834584 (+ 3003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:11.834589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:11.834823 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:11.834984 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:11.836731 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:11.837982 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:11.840257 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:11.840901 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:11.841035 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:11.844939 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:11.844957 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:11.844980 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:11.845036 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:11.849306 (+ 4270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:11.850032 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:11.850045 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:11.850051 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:11.850303 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:11.850713 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:11.850721 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:11.850727 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:11.851469 (+ 742us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:11.851683 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:11.852249 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 09:07:11.852733 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":652,"thread_start_us":414,"threads_started":1}
I20260811 09:07:11.856739 2913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:11.857860 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.857447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51448)
0811 09:07:11.857631 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.857747 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:07:11.858150 2850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
I20260811 09:07:11.859735 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.859403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51464)
0811 09:07:11.859553 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.859626 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 09:07:11.860000 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:40727: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260811 09:07:11.906034 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:40727
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:07:11.909361 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } attempt: 1
I20260811 09:07:11.910581 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.910136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51466)
0811 09:07:11.910340 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.910429 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 09:07:11.911620 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.911266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51482)
0811 09:07:11.911422 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.911487 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 09:07:11.911789 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:40727: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
I20260811 09:07:11.980105 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } attempt: 2
I20260811 09:07:11.981449 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.980950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51490)
0811 09:07:11.981211 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.981302 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 09:07:11.982506 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:11.982130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51498)
0811 09:07:11.982299 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:11.982380 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 09:07:11.982705 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:40727: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
I20260811 09:07:12.073999 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } attempt: 3
I20260811 09:07:12.075275 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.074750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51514)
0811 09:07:12.074966 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.075069 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:07:12.076385 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.076015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51520)
0811 09:07:12.076176 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.076251 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:07:12.076561 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:40727: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
I20260811 09:07:12.219944 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } attempt: 4
I20260811 09:07:12.221299 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.220763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51536)
0811 09:07:12.221073 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.221166 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260811 09:07:12.222388 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.222018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51550)
0811 09:07:12.222189 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.222266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 09:07:12.222560 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:40727: Network error: Client connection negotiation failed: client connection to 127.9.40.125:40727: connect: Connection refused (error 111)
W20260811 09:07:12.259931 2923 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:12.260345 2923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:12.260481 2923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:12.260627 2923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:12.267985 2923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:12.268152 2923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:12.268280 2923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:12.268373 2923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:12.268472 2923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:12.279078 2923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:40727
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:12.280514 2923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:12.281898 2923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:12.294181 2928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:12.294160 2929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:12.294153 2931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:12.294956 2923 server_base.cc:1034] running on GCE node
I20260811 09:07:12.295744 2923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:12.297360 2923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:12.298590 2923 hybrid_clock.cc:648] HybridClock initialized: now 1786439232298526 us; error 35 us; skew 500 ppm
I20260811 09:07:12.298961 2923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:12.302436 2923 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260811 09:07:12.304010 2923 webserver.cc:460] Webserver started at http://127.9.40.125:38785/ using document root <none> and password file <none>
I20260811 09:07:12.304829 2923 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:12.304956 2923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:12.305328 2923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:12.308609 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data/instance:
uuid: "dea212c27f97438d9ced3dd2ca40a3ba"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "487a65185685c782de2e774b705911ac"
server_key_iv: "a370e8b6e6a80575204b98aff33c08b5"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.309543 2923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal/instance:
uuid: "dea212c27f97438d9ced3dd2ca40a3ba"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "487a65185685c782de2e774b705911ac"
server_key_iv: "a370e8b6e6a80575204b98aff33c08b5"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.315127 2923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:12.319453 2938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:12.320994 2923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:07:12.321210 2923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "dea212c27f97438d9ced3dd2ca40a3ba"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "487a65185685c782de2e774b705911ac"
server_key_iv: "a370e8b6e6a80575204b98aff33c08b5"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.321453 2923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:12.342196 2923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:12.346499 2923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:12.346807 2923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:12.366380 2923 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:40727
I20260811 09:07:12.366395 2999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:40727 every 8 connection(s)
I20260811 09:07:12.367815 2923 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
I20260811 09:07:12.371745 3000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:12.373668 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 2923
I20260811 09:07:12.373873 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-1/wal/instance
I20260811 09:07:12.374471 9377 external_mini_cluster.cc:1428] Setting key 62504f327cafeda8f4045d615a733b86
I20260811 09:07:12.382267 3000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:12.408968 3002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.386008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47158 (local address 127.9.40.126:44941)
0811 09:07:12.386584 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.386590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.387856 (+ 1266us) server_negotiation.cc:407] Connection header received
0811 09:07:12.390991 (+ 3135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.390999 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.391098 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.391221 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.393201 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.394449 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.396087 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.396864 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.397032 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.400822 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.400834 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.400842 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.400921 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.404993 (+ 4072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.405876 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.405885 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.405891 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.406052 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.406561 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.406569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.406575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.407047 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.407218 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.408157 (+ 939us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.408434 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":288,"threads_started":1}
I20260811 09:07:12.409806 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.386609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44941 (local address 127.0.0.1:47158)
0811 09:07:12.387586 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.387651 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.390572 (+ 2921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.391541 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.391556 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.392235 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.392970 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.392992 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.394644 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.394648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.395926 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.395940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.397103 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.397108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.397360 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.398121 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.398149 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.400502 (+ 2353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.405186 (+ 4684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.405195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.405212 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.405732 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.406186 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.406190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.406192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.406397 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.407220 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.407227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.407757 (+ 530us) client_negotiation.cc:769] Sending connection context
0811 09:07:12.408026 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:07:12.408462 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":405,"threads_started":1}
I20260811 09:07:12.411685 3000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:12.427266 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.412554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51554)
0811 09:07:12.412835 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.412855 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.412974 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.414065 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.414069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.414555 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.414697 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.414705 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.416291 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.416299 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.417267 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.417279 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.418134 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.418139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.418356 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.418994 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.419012 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.421145 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.424390 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.424398 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.424402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.424866 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.425319 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.425325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.425328 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.425481 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.426194 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.426224 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.426369 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 09:07:12.426516 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:07:12.426851 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":1}
I20260811 09:07:12.427352 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.412835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51554 (local address 127.9.40.125:40727)
0811 09:07:12.413612 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.413621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.413647 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:07:12.413692 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.413697 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.413878 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.414071 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.414883 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.416157 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.417417 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.417983 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.418132 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.421355 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.421367 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.421382 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.421419 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.424254 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.425011 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.425018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.425020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.425208 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.425618 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.425623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.425626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.426058 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.426308 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.426682 (+ 374us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.426966 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":92,"server-negotiator.queue_time_us":618,"thread_start_us":460,"threads_started":1}
I20260811 09:07:12.430045 3000 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:12.431295 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.430801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53030)
0811 09:07:12.431002 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.431132 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 09:07:12.431595 2939 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.433259 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.432913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53032)
0811 09:07:12.433071 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.433151 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:07:12.433501 3000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260811 09:07:12.455965 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:36647
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:07:12.482909 3000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 1
I20260811 09:07:12.484361 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.483701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53038)
0811 09:07:12.483986 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.484140 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 09:07:12.485778 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.485241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53046)
0811 09:07:12.485466 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.485585 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:07:12.486013 3000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.521953 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } attempt: 5
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:12.536095 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.522947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51560 (local address 127.9.40.125:40727)
0811 09:07:12.523137 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.523145 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.523253 (+ 108us) server_negotiation.cc:407] Connection header received
0811 09:07:12.523485 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.523490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.523553 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.523665 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.524501 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.525392 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.526740 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.527205 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.527345 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.530636 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.530641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.530644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.530678 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.533405 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.534104 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.534112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.534121 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.534242 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.534634 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.534639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.534643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.534948 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.535112 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.535526 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.535764 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:07:12.536134 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.522878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51560)
0811 09:07:12.523121 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.523145 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.523323 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.523703 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.523708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.524144 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.524343 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.524353 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.525534 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.525541 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.526590 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.526604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.527359 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.527365 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.527576 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.528241 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.528257 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.530420 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.533554 (+ 3134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.533559 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.533564 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.533970 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.534360 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.534365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.534369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.534515 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.535079 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.535084 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.535255 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 09:07:12.535402 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 09:07:12.535753 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:07:12.537252 2913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:12.538316 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.537921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53052)
0811 09:07:12.538094 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.538172 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 09:07:12.539268 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.538945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53064)
0811 09:07:12.539096 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.539175 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:07:12.539492 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.554276 3000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 2
I20260811 09:07:12.555531 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.554972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53076)
0811 09:07:12.555210 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.555328 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:07:12.556634 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.556242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53090)
0811 09:07:12.556396 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.556479 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 09:07:12.556872 3000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.590791 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 1
I20260811 09:07:12.592145 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.591542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53098)
0811 09:07:12.591806 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.591925 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 09:07:12.593303 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.592880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53110)
0811 09:07:12.593062 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.593156 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 09:07:12.593529 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.648119 3000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 3
I20260811 09:07:12.649505 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.648920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53112)
0811 09:07:12.649167 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.649297 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 09:07:12.650790 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.650249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53118)
0811 09:07:12.650516 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.650635 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
W20260811 09:07:12.651039 3000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.661785 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 2
I20260811 09:07:12.663123 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.662532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53126)
0811 09:07:12.662794 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.662918 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 09:07:12.664197 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.663782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53138)
0811 09:07:12.663941 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.664035 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:07:12.664456 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.770733 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 3
I20260811 09:07:12.772061 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.771460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53152)
0811 09:07:12.771721 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.771844 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 09:07:12.773247 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.772810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53158)
0811 09:07:12.772996 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.773099 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:07:12.773499 2913 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
I20260811 09:07:12.794301 3000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 4
I20260811 09:07:12.795765 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.795078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53168)
0811 09:07:12.795387 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.795516 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 09:07:12.796980 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.796534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53180)
0811 09:07:12.796739 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.796818 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:07:12.797271 3000 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:36647: Network error: Client connection negotiation failed: client connection to 127.9.40.124:36647: connect: Connection refused (error 111)
W20260811 09:07:12.829128 3009 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:12.829456 3009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:12.829533 3009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:12.829617 3009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:12.837167 3009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:12.837294 3009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:12.837354 3009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:12.837407 3009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:12.837464 3009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:12.847565 3009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44941,127.9.40.125:40727,127.9.40.124:36647
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:36647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:12.849026 3009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:12.850396 3009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:12.863183 3015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:12.863135 3016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:12.863194 3018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:12.864120 3009 server_base.cc:1034] running on GCE node
I20260811 09:07:12.865044 3009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:12.866515 3009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:12.867781 3009 hybrid_clock.cc:648] HybridClock initialized: now 1786439232867689 us; error 54 us; skew 500 ppm
I20260811 09:07:12.868114 3009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:12.871304 3009 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260811 09:07:12.872900 3009 webserver.cc:460] Webserver started at http://127.9.40.124:38885/ using document root <none> and password file <none>
I20260811 09:07:12.873618 3009 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:12.873775 3009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:12.874135 3009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:12.877449 3009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data/instance:
uuid: "bc34a924d50c408b9394af236b7e0284"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "7d7f2122c24cb472897b5d921c3bafe8"
server_key_iv: "5a2c3d2c447938957311d74069604a71"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.878293 3009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal/instance:
uuid: "bc34a924d50c408b9394af236b7e0284"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "7d7f2122c24cb472897b5d921c3bafe8"
server_key_iv: "5a2c3d2c447938957311d74069604a71"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.883801 3009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:07:12.887326 3026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:12.888804 3009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:07:12.888983 3009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "bc34a924d50c408b9394af236b7e0284"
format_stamp: "Formatted at 2026-08-11 09:07:12 on dist-test-slave-cbsf"
server_key: "7d7f2122c24cb472897b5d921c3bafe8"
server_key_iv: "5a2c3d2c447938957311d74069604a71"
server_key_version: "encryptionkey@0"
I20260811 09:07:12.889230 3009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:12.912326 3009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:12.916635 3009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:12.916960 3009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:12.936434 3009 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:36647
I20260811 09:07:12.936453 3089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:36647 every 8 connection(s)
I20260811 09:07:12.937881 3009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
I20260811 09:07:12.941782 3090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:12.943897 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 3009
I20260811 09:07:12.944114 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786438852478101-9377-0/minicluster-data/master-2/wal/instance
I20260811 09:07:12.944720 9377 external_mini_cluster.cc:1428] Setting key 57550b08e8669e58a35177b8361185c2
I20260811 09:07:12.950732 2913 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 4
I20260811 09:07:12.952234 3090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:12.971263 2917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.951368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53202)
0811 09:07:12.951544 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.951564 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.951661 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.952415 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.952422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.953018 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.953239 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.953253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.955352 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.955358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.956912 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.956930 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.959480 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.959484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.959605 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.960197 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.960223 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.963421 (+ 3198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.967787 (+ 4366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.967792 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.967796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.968241 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.968771 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.968777 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.968781 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.968980 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.970128 (+ 1148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.970133 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.970312 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:07:12.970496 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 09:07:12.970853 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:07:12.972847 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.951516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.9.40.124:36647)
0811 09:07:12.952135 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.952140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.952157 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:12.952193 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.952196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.952245 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.952362 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.953460 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.955137 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.957129 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.959310 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.959438 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.963650 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.963658 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.963661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.963699 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.967604 (+ 3905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.968399 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.968415 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.968443 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.968623 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.969131 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.969138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.969145 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.969919 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.970138 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.970811 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.971442 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":472,"server-negotiator.queue_time_us":501,"spinlock_wait_cycles":19328,"thread_start_us":346,"threads_started":1}
I20260811 09:07:12.972874 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.946749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53196 (local address 127.9.40.124:36647)
0811 09:07:12.947626 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.947638 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.947684 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:07:12.949387 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.949414 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.949800 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.950119 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.952821 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.955137 (+ 2316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.957129 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.958343 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.958526 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.963391 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.963401 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.963415 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.963457 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.968009 (+ 4552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.968903 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.968908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.968911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.969072 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.969515 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.969520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.969523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.969919 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.970312 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.970778 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.971334 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":153,"server-negotiator.queue_time_us":651,"thread_start_us":437,"threads_started":1}
I20260811 09:07:12.976958 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.954997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44941 (local address 127.0.0.1:47166)
0811 09:07:12.958097 (+ 3100us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.958147 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.958296 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.959882 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.959887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.960829 (+ 942us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.961143 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.961155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.962443 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.962453 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.963835 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.963854 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.964910 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.964917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.965152 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.966153 (+ 1001us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.966198 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.969320 (+ 3122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.973630 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.973640 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.973646 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.974213 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.974762 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.974769 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.974775 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.974943 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.975763 (+ 820us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.975770 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.975957 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 09:07:12.976164 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 09:07:12.976491 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2859,"thread_start_us":2712,"threads_started":1}
I20260811 09:07:12.976969 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.955000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47166 (local address 127.9.40.126:44941)
0811 09:07:12.958949 (+ 3949us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.958954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.959000 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:07:12.959053 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.959058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.959127 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.959238 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.961318 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.962276 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.964004 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.964672 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.964825 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.969537 (+ 4712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.969543 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.969546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.969575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.973535 (+ 3960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.974353 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.974358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.974361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.974523 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.975088 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.975093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.975107 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.975537 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.975689 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.976276 (+ 587us) server_negotiation.cc:299] Negotiation successful
0811 09:07:12.976491 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3829,"thread_start_us":3691,"threads_started":1}
I20260811 09:07:12.979779 2913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: Bootstrap starting.
I20260811 09:07:12.982179 3090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:12.984508 2913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:12.985797 2913 log.cc:824] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:12.989923 2913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: No bootstrap required, opened a new log
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
Aug 11 09:07:12 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:13.000677 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.983100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.9.40.125:40727)
0811 09:07:12.983323 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.983332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.983354 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:07:12.983547 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.983552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.983626 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.983731 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.984429 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.985512 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.987197 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.987887 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.987992 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.993256 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.993264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.993269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.993319 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.997226 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.998174 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.998180 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.998184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.998348 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:12.998864 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:12.998869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:12.998873 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:12.999237 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:12.999362 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:12.999995 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.000252 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:07:13.001652 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.983080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:40727 (local address 127.0.0.1:51570)
0811 09:07:12.983299 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:12.983321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:12.983414 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:12.983757 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:12.983761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:12.984151 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:12.984283 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.984291 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.985681 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.985686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.986997 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:12.987028 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.988266 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.988272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:12.988461 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.989129 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:12.989152 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:12.992999 (+ 3847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:12.997406 (+ 4407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.997411 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.997415 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.998015 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.998518 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:12.998524 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:12.998528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:12.998719 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:12.999470 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:12.999475 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:12.999658 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 09:07:13.000908 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0811 09:07:13.001242 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:07:13.002863 3090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:13.002930 2913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.003600 2913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2b58f65d84c59a491b3a6d8cea674, State: Initialized, Role: FOLLOWER
I20260811 09:07:13.003973 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:12.981325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47174 (local address 127.9.40.126:44941)
0811 09:07:12.981521 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:12.981527 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:12.981545 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:07:12.981611 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:12.981614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:12.981694 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:12.981815 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:12.983361 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.984472 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.986175 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:12.986942 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:12.987110 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:12.992371 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:12.992468 (+ 97us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:12.992478 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:12.992550 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:12.998961 (+ 6411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.000344 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.000349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.000383 (+ 34us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.000641 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.001396 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.001440 (+ 44us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.001444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.002160 (+ 716us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.002503 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.003353 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.003643 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:07:13.004225 2913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.005434 3098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.005959 3098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.006968 2913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 09:07:13 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:07:13 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:13.020017 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.003725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53216)
0811 09:07:13.004041 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:13.004065 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:13.004783 (+ 718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:13.005292 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:13.005296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:13.005840 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:13.006043 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:13.006056 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.007279 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.007284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:13.008534 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:13.008545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.009605 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.009610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:13.009786 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.010419 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:13.010448 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:13.013343 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:13.017304 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:13.017312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:13.017316 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:13.017692 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:13.018130 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:13.018135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:13.018138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:13.018277 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:13.018833 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:13.018837 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:13.018980 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 09:07:13.019137 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 09:07:13.019512 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":21}
I20260811 09:07:13.020404 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.004040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53216 (local address 127.9.40.124:36647)
0811 09:07:13.004227 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.004233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.004252 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:13.004928 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.004936 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.005011 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.005081 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.006190 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.007119 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.008667 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.009444 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.009593 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.013540 (+ 3947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.013549 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.013553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.013593 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.017177 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.017812 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.017820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.017828 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.018013 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.018384 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.018390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.018393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.018719 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.018871 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.019218 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.019677 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":62}
I20260811 09:07:13.027010 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.006333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.9.40.125:40727)
0811 09:07:13.006523 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.006529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.006547 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:07:13.006597 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.006602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.006668 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.006779 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.008340 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.009912 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.012040 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.012789 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.012888 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.019331 (+ 6443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.019337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.019342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.019385 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.023343 (+ 3958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.024319 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.024325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.024328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.024515 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.025037 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.025042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.025047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.025401 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.025698 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.026294 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.026632 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:07:13.027431 3090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: Bootstrap starting.
I20260811 09:07:13.032719 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:13.032677 3090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:13.034020 3090 log.cc:824] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:13.025138 2834 master_runner.cc:186] Error getting master registration for 127.9.40.125:40727: OK, instance_id { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" instance_seqno: 1786439232360804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:07:13.036921 2881 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6fe2b58f65d84c59a491b3a6d8cea674"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47174
I20260811 09:07:13.037381 2834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:07:13.037636 2881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:07:13.037885 2881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:13.042959 2881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
W20260811 09:07:13.043530 3111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:07:13.043692 3111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:07:13.044900 2881 leader_election.cc:288] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [CANDIDATE]: Term 1 election: Requested vote from peers dea212c27f97438d9ced3dd2ca40a3ba (127.9.40.125:40727), bc34a924d50c408b9394af236b7e0284 (127.9.40.124:36647)
W20260811 09:07:13.045958 2855 tablet.cc:2369] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:13.046988 3090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: No bootstrap required, opened a new log
I20260811 09:07:13.058699 3090 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.059214 3090 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc34a924d50c408b9394af236b7e0284, State: Initialized, Role: FOLLOWER
I20260811 09:07:13.059814 3090 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.062650 3090 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:13.062760 3063 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bc34a924d50c408b9394af236b7e0284"
I20260811 09:07:13.063364 3063 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:13.068055 3063 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2b58f65d84c59a491b3a6d8cea674 in term 1.
I20260811 09:07:13.068931 2850 leader_election.cc:302] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2b58f65d84c59a491b3a6d8cea674, bc34a924d50c408b9394af236b7e0284; no voters:
I20260811 09:07:13.069536 3098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:13.061740 3117 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.091956 3117 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.096505 3000 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } attempt: 5
I20260811 09:07:13.102264 3005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.070500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51590 (local address 127.9.40.125:40727)
0811 09:07:13.070682 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.070688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.072029 (+ 1341us) server_negotiation.cc:407] Connection header received
0811 09:07:13.072350 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.072356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.072446 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.072519 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.077252 (+ 4733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.078567 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.083035 (+ 4468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.083924 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.084054 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.091993 (+ 7939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.092002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.092010 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.092064 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.096536 (+ 4472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.098568 (+ 2032us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.098574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.098578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.098766 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.099727 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.099735 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.099740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.100190 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.100481 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.101592 (+ 1111us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.101839 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:07:13.106652 3098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [term 1 LEADER]: Becoming Leader. State: Replica: 6fe2b58f65d84c59a491b3a6d8cea674, State: Running, Role: LEADER
I20260811 09:07:13.108299 3098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
Aug 11 09:07:13 dist-test-slave-cbsf krb5kdc[2821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439232, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:07:13.126660 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.072377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.9.40.124:36647)
0811 09:07:13.072539 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.072543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.103594 (+ 31051us) server_negotiation.cc:407] Connection header received
0811 09:07:13.104819 (+ 1225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.104824 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.104905 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.104982 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.108362 (+ 3380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.109445 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.111984 (+ 2539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.112792 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.112893 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.116631 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.116641 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.116647 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.116685 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.121110 (+ 4425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.122095 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.122113 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.122118 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.122419 (+ 301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.123010 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.123018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.123024 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.123800 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.125069 (+ 1269us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.125391 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.125862 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56,"spinlock_wait_cycles":1920}
I20260811 09:07:13.128160 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.098290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53240 (local address 127.9.40.124:36647)
0811 09:07:13.098552 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.098561 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.098587 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:07:13.098668 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.098676 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.098776 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.098896 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.100555 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.102118 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.104535 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.105513 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.105660 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.111660 (+ 6000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.111673 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.111682 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.111737 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.116893 (+ 5156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.122127 (+ 5234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.122135 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.122142 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.122389 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.125235 (+ 2846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.125243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.125249 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.125841 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.126074 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.127308 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.127641 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78,"spinlock_wait_cycles":72576}
I20260811 09:07:13.129519 3003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.097356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:36647 (local address 127.0.0.1:53240)
0811 09:07:13.097836 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:13.097861 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:13.098018 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:13.099175 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:13.099184 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:13.099907 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:13.100223 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:13.100239 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.102337 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.102345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:13.104318 (+ 1973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:13.104337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.105895 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.105904 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:13.106196 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.107044 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:13.107075 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:13.111266 (+ 4191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:13.117183 (+ 5917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:13.117191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:13.117198 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:13.121922 (+ 4724us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:13.124758 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:13.124768 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:13.124775 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:13.125050 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:13.126174 (+ 1124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:13.126182 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:13.126408 (+ 226us) client_negotiation.cc:769] Sending connection context
0811 09:07:13.126593 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 09:07:13.128903 (+ 2310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":4}
I20260811 09:07:13.131500 3120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fe2b58f65d84c59a491b3a6d8cea674. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.131953 3120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:13.135298 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:13.070033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.9.40.126:44941)
0811 09:07:13.070248 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.070256 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.101262 (+ 31006us) server_negotiation.cc:407] Connection header received
0811 09:07:13.102125 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.102131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.102224 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.102303 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.116021 (+ 13718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.117294 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.119769 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:13.120470 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:13.120571 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:13.124494 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:13.124502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:13.124508 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:13.124565 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:13.130128 (+ 5563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.132108 (+ 1980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.132116 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.132122 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.132304 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:13.132903 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:13.132909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:13.132913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:13.133397 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:13.133755 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:13.134470 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 09:07:13.134797 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:07:13.138566 3000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: Bootstrap starting.
I20260811 09:07:13.144171 3000 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:13.154297 3000 log.cc:824] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:13.155741 3128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:13.158643 3128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:13.163511 3000 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: No bootstrap required, opened a new log
W20260811 09:07:13.169149 3032 tablet.cc:2369] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:13.178081 3063 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [term 1 FOLLOWER]: Refusing update from remote peer 6fe2b58f65d84c59a491b3a6d8cea674: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:13.180011 3120 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:13.182313 3000 raft_consensus.cc:348] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.182873 3000 raft_consensus.cc:729] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea212c27f97438d9ced3dd2ca40a3ba, State: Initialized, Role: FOLLOWER
I20260811 09:07:13.183533 3000 consensus_queue.cc:260] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } }
I20260811 09:07:13.216348 3117 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fe2b58f65d84c59a491b3a6d8cea674. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.220501 3117 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.224627 3000 sys_catalog.cc:564] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:13.224721 2973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "dea212c27f97438d9ced3dd2ca40a3ba"
I20260811 09:07:13.225143 2972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:13.229872 3131 mvcc.cc:205] Tried to move back new op lower bound from 7317255099081875456 to 7317255098826285056. Current Snapshot: MvccSnapshot[applied={T|T < 7317255099081875456}]
I20260811 09:07:13.230059 2972 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [term 1 FOLLOWER]: Refusing update from remote peer 6fe2b58f65d84c59a491b3a6d8cea674: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:13.231441 3136 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.231979 3136 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.236752 2973 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2b58f65d84c59a491b3a6d8cea674 in term 1.
I20260811 09:07:13.239185 3098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:13.241385 3117 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.241669 3117 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.251780 3098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.252134 3098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:13.253130 3128 catalog_manager.cc:1347] Generated new cluster ID: ae387bafeded4e55b8891d86aae35381
I20260811 09:07:13.253232 3128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:13.281169 3136 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fe2b58f65d84c59a491b3a6d8cea674. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.281719 3136 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.281340 3152 sys_catalog.cc:455] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.282655 3152 sys_catalog.cc:458] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:13.283638 3120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2b58f65d84c59a491b3a6d8cea674" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44941 } } peers { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 40727 } } peers { permanent_uuid: "bc34a924d50c408b9394af236b7e0284" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 36647 } } }
I20260811 09:07:13.284025 3120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:13.289682 3128 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:13.291776 3128 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:13.326188 3128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fe2b58f65d84c59a491b3a6d8cea674: Generated new TSK 0
I20260811 09:07:13.327582 3128 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:07:13.349987 3165 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:07:13.354614 3162 catalog_manager.cc:1259] Loaded cluster ID: ae387bafeded4e55b8891d86aae35381
I20260811 09:07:13.354732 3162 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: loading cluster ID for follower catalog manager: success
I20260811 09:07:13.341310 3009 master_runner.cc:186] Error getting master registration for 127.9.40.125:40727: OK, instance_id { permanent_uuid: "dea212c27f97438d9ced3dd2ca40a3ba" instance_seqno: 1786439232360804 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:07:13.361162 3009 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:07:13.363628 3162 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: acquiring CA information for follower catalog manager: success
I20260811 09:07:13.378007 3162 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bc34a924d50c408b9394af236b7e0284: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:07:13.456779 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.446890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51592 (local address 127.9.40.125:40727)
0811 09:07:13.447366 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.447372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.454425 (+ 7053us) server_negotiation.cc:407] Connection header received
0811 09:07:13.454961 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.454967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.455074 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.455158 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.456472 (+ 1314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51592: BlockingRecv error: recv got EOF from 127.0.0.1:51592 (error 108)
Metrics: {"server-negotiator.queue_time_us":340}
W20260811 09:07:13.459452 3095 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.446664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47196 (local address 127.9.40.126:44941)
0811 09:07:13.447166 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.447173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.457281 (+ 10108us) server_negotiation.cc:407] Connection header received
0811 09:07:13.457567 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.457573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.457666 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.457739 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.459201 (+ 1462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47196: BlockingRecv error: recv got EOF from 127.0.0.1:47196 (error 108)
Metrics: {"server-negotiator.queue_time_us":311}
W20260811 09:07:13.460192 3094 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.457889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53248 (local address 127.9.40.124:36647)
0811 09:07:13.458096 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.458102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.458572 (+ 470us) server_negotiation.cc:407] Connection header received
0811 09:07:13.458818 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.458823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.458948 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.459019 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.459894 (+ 875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53248: BlockingRecv error: recv got EOF from 127.0.0.1:53248 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 09:07:13.468142 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.465757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.9.40.125:40727)
0811 09:07:13.465946 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.465953 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.466525 (+ 572us) server_negotiation.cc:407] Connection header received
0811 09:07:13.466765 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.466771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.466906 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.467030 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.467844 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51594: BlockingRecv error: recv got EOF from 127.0.0.1:51594 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:07:13.488874 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.487147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51598 (local address 127.9.40.125:40727)
0811 09:07:13.487328 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.487335 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.487355 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:07:13.487440 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.487446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.487545 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.487655 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.488597 (+ 942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51598: BlockingRecv error: recv got EOF from 127.0.0.1:51598 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 09:07:13.497069 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.494705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51606 (local address 127.9.40.125:40727)
0811 09:07:13.494894 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.494900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.495444 (+ 544us) server_negotiation.cc:407] Connection header received
0811 09:07:13.495635 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.495641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.495751 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.495864 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.496790 (+ 926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51606: BlockingRecv error: recv got EOF from 127.0.0.1:51606 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:07:13.507179 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.504615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51618 (local address 127.9.40.125:40727)
0811 09:07:13.504892 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.504901 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.505528 (+ 627us) server_negotiation.cc:407] Connection header received
0811 09:07:13.505764 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.505771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.505867 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.505991 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.506935 (+ 944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51618: BlockingRecv error: recv got EOF from 127.0.0.1:51618 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
W20260811 09:07:13.525444 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.522984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.9.40.125:40727)
0811 09:07:13.523195 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.523202 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.523760 (+ 558us) server_negotiation.cc:407] Connection header received
0811 09:07:13.523981 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.523988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.524088 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.524199 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.525175 (+ 976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51630: BlockingRecv error: recv got EOF from 127.0.0.1:51630 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 09:07:13.549201 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.546401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.9.40.125:40727)
0811 09:07:13.546611 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.546618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.547196 (+ 578us) server_negotiation.cc:407] Connection header received
0811 09:07:13.547519 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.547526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.547631 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.547720 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.548939 (+ 1219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51632: BlockingRecv error: recv got EOF from 127.0.0.1:51632 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 09:07:13.589342 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.586704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.9.40.125:40727)
0811 09:07:13.586917 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.586925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.587506 (+ 581us) server_negotiation.cc:407] Connection header received
0811 09:07:13.587853 (+ 347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.587860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.587973 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.588088 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.589060 (+ 972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51646: BlockingRecv error: recv got EOF from 127.0.0.1:51646 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 09:07:13.667629 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.665291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51656 (local address 127.9.40.125:40727)
0811 09:07:13.665480 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.665487 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.666052 (+ 565us) server_negotiation.cc:407] Connection header received
0811 09:07:13.666268 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.666277 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.666386 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.666513 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.667397 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51656: BlockingRecv error: recv got EOF from 127.0.0.1:51656 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 09:07:13.801121 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:13.799285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51670 (local address 127.9.40.125:40727)
0811 09:07:13.799472 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:13.799477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:13.799841 (+ 364us) server_negotiation.cc:407] Connection header received
0811 09:07:13.800003 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:13.800008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:13.800081 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:13.800181 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:13.800909 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51670: BlockingRecv error: recv got EOF from 127.0.0.1:51670 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:07:14.063158 3005 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:14.061196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51678 (local address 127.9.40.125:40727)
0811 09:07:14.061382 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.061387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.061634 (+ 247us) server_negotiation.cc:407] Connection header received
0811 09:07:14.061852 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.061857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.061937 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.062041 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:14.062891 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51678: BlockingRecv error: recv got EOF from 127.0.0.1:51678 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:07:14.354526 3165 catalog_manager.cc:1259] Loaded cluster ID: ae387bafeded4e55b8891d86aae35381
I20260811 09:07:14.354714 3165 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: loading cluster ID for follower catalog manager: success
I20260811 09:07:14.357468 3165 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: acquiring CA information for follower catalog manager: success
I20260811 09:07:14.359315 3165 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dea212c27f97438d9ced3dd2ca40a3ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:07:14.588018 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:14.579276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51692 (local address 127.9.40.125:40727)
0811 09:07:14.581595 (+ 2319us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.581599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.581624 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:07:14.581695 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.581699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.581858 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.581979 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:14.582511 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.583405 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.585422 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.585887 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.585985 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:14.586649 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:14.587240 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:14.587349 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:14.587520 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 09:07:14.587594 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2197,"thread_start_us":2015,"threads_started":1}
I20260811 09:07:14.602135 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:14.595440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53256 (local address 127.9.40.124:36647)
0811 09:07:14.596132 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.596138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.596155 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:14.596255 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.596259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.596414 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.596557 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:14.597096 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.597841 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.599784 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.600183 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.600273 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:14.600967 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:14.601472 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:14.601589 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:14.601748 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 09:07:14.601818 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":462,"threads_started":1}
I20260811 09:07:14.620728 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:14.610433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47212 (local address 127.9.40.126:44941)
0811 09:07:14.610939 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.610945 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.611892 (+ 947us) server_negotiation.cc:407] Connection header received
0811 09:07:14.612283 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.612289 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.612515 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.612686 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:14.613529 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.614645 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.616937 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.617594 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.617708 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:14.618501 (+ 793us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:14.619101 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:14.619232 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:14.620229 (+ 997us) server_negotiation.cc:299] Negotiation successful
0811 09:07:14.620324 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
I20260811 09:07:14.626119 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:14.612432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.9.40.124:36647)
0811 09:07:14.612654 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.612662 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.620044 (+ 7382us) server_negotiation.cc:407] Connection header received
0811 09:07:14.620207 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.620212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.620297 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.620380 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:14.621176 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.622067 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.623645 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.624312 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.624431 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:14.625146 (+ 715us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:14.625280 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:14.625344 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:14.625642 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:07:14.625735 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:07:14.626132 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2834
I20260811 09:07:14.627384 2906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:14.627866 3245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:14.610190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51696 (local address 127.9.40.125:40727)
0811 09:07:14.610410 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:14.610417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:14.618407 (+ 7990us) server_negotiation.cc:407] Connection header received
0811 09:07:14.618615 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:14.618621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:14.618715 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:14.618815 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:14.619669 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.620861 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.625099 (+ 4238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:14.625931 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:14.626028 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:14.626974 (+ 946us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:14.627126 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:14.627222 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:14.627480 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 09:07:14.627563 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:07:14.745546 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2834
W20260811 09:07:14.773412 2942 connection.cc:535] server connection from 127.0.0.1:51560 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:07:14.774138 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 2923
I20260811 09:07:14.775313 2995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:14.886346 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 2923
I20260811 09:07:14.914222 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 3009
I20260811 09:07:14.915382 3085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:15.030375 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 3009
2026-08-11T09:07:15Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5290 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:15 dist-test-slave-cbsf krb5kdc[3269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:15 dist-test-slave-cbsf krb5kdc[3269](info): set up 2 sockets
Aug 11 09:07:15 dist-test-slave-cbsf krb5kdc[3269](info): commencing operation
krb5kdc: starting...
W20260811 09:07:16.260284 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.003s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/test-user.keytab.
Aug 11 09:07:16 dist-test-slave-cbsf krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260811 09:07:16.623883 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43033
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:17.005115 3282 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:17.005621 3282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:17.005715 3282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:17.005807 3282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:17.017781 3282 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:17.017941 3282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:17.018136 3282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:17.018236 3282 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:17.018352 3282 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:17.031992 3282 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:17.033505 3282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:17.034816 3282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:17.047124 3287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:17.047060 3288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:17.047015 3293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:17.048175 3282 server_base.cc:1034] running on GCE node
I20260811 09:07:17.049068 3282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:17.050451 3282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:17.051707 3282 hybrid_clock.cc:648] HybridClock initialized: now 1786439237051632 us; error 35 us; skew 500 ppm
I20260811 09:07:17.052102 3282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:17.055495 3282 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260811 09:07:17.057202 3282 webserver.cc:460] Webserver started at http://127.9.40.126:40587/ using document root <none> and password file <none>
I20260811 09:07:17.057932 3282 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:17.058075 3282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:17.058411 3282 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:17.061702 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data/instance:
uuid: "de2452320b5e4e88935df891f20c6d8b"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "65a85332c86ffa1f2758704913ac1f80"
server_key_iv: "c1a08cb0ce935aa0da2fbc36cd430e3c"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.062553 3282 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance:
uuid: "de2452320b5e4e88935df891f20c6d8b"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "65a85332c86ffa1f2758704913ac1f80"
server_key_iv: "c1a08cb0ce935aa0da2fbc36cd430e3c"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.067977 3282 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:07:17.071552 3300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:17.073076 3282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:17.073280 3282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal
uuid: "de2452320b5e4e88935df891f20c6d8b"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "65a85332c86ffa1f2758704913ac1f80"
server_key_iv: "c1a08cb0ce935aa0da2fbc36cd430e3c"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.073519 3282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:17.095336 3282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:17.099853 3282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:17.100155 3282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:17.120222 3282 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43033
I20260811 09:07:17.120249 3361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43033 every 8 connection(s)
I20260811 09:07:17.121703 3282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:17.125460 3363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:17.130342 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 3282
I20260811 09:07:17.130551 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-0/wal/instance
I20260811 09:07:17.131007 9377 external_mini_cluster.cc:1428] Setting key 4f827918e245d0350d725a63398635aa
I20260811 09:07:17.132507 3363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:17.156211 3366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.135850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44078 (local address 127.9.40.126:43033)
0811 09:07:17.136733 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:17.136752 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:17.137184 (+ 432us) server_negotiation.cc:407] Connection header received
0811 09:07:17.139180 (+ 1996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:17.139186 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:17.139416 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:17.139622 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:17.140992 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.142268 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.143657 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.144212 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.144414 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.148398 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:17.148406 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:17.148418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:17.148449 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:17.151291 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.152232 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.152239 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.152241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.152385 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.152897 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.152900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.152902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.153423 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:17.153584 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:17.154275 (+ 691us) server_negotiation.cc:299] Negotiation successful
0811 09:07:17.154880 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":607,"thread_start_us":384,"threads_started":1}
I20260811 09:07:17.156253 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.135850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43033 (local address 127.0.0.1:44078)
0811 09:07:17.137011 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.137046 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:17.138814 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:17.139649 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:17.139656 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:17.140379 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:17.140779 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.140805 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.142432 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.142438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.143511 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.143524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.144450 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.144457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.144649 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.145388 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:17.145411 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:17.148158 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:17.151434 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.151461 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.151475 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.152085 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.152494 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.152501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.152507 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.152772 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.153578 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:17.153589 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:17.153856 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 09:07:17.154149 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 09:07:17.155173 (+ 1024us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":287,"thread_start_us":748,"threads_started":1}
I20260811 09:07:17.159220 3363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:17.160559 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.159980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:59994)
0811 09:07:17.160213 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.160399 (+ 186us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:07:17.160897 3302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
I20260811 09:07:17.162693 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.162206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60002)
0811 09:07:17.162417 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.162516 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:07:17.162930 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:41843: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260811 09:07:17.211752 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:41843
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:07:17.212271 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } attempt: 1
I20260811 09:07:17.213488 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.213050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60018)
0811 09:07:17.213251 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.213342 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:07:17.214447 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.214115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60024)
0811 09:07:17.214255 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.214325 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:07:17.214627 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:41843: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
I20260811 09:07:17.282896 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } attempt: 2
I20260811 09:07:17.284216 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.283610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60028)
0811 09:07:17.283905 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.284022 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:07:17.285315 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.284902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60040)
0811 09:07:17.285047 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.285157 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:07:17.285547 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:41843: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
I20260811 09:07:17.376845 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } attempt: 3
I20260811 09:07:17.378279 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.377745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60044)
0811 09:07:17.377967 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.378092 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 09:07:17.379401 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.378963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60054)
0811 09:07:17.379122 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.379209 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:07:17.379642 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:41843: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
I20260811 09:07:17.522917 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } attempt: 4
I20260811 09:07:17.524327 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.523643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60060)
0811 09:07:17.524018 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.524139 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 09:07:17.525612 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.525111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60062)
0811 09:07:17.525307 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.525418 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:07:17.525870 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:41843: Network error: Client connection negotiation failed: client connection to 127.9.40.125:41843: connect: Connection refused (error 111)
W20260811 09:07:17.568436 3371 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:17.568805 3371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:17.568876 3371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:17.568964 3371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:17.576489 3371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:17.576593 3371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:17.576653 3371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:17.576748 3371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:17.576833 3371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:17.587060 3371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:41843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:17.588397 3371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:17.589823 3371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:17.602030 3377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:17.602140 3376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:17.602694 3379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:17.602944 3371 server_base.cc:1034] running on GCE node
I20260811 09:07:17.603770 3371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:17.605206 3371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:17.606428 3371 hybrid_clock.cc:648] HybridClock initialized: now 1786439237606370 us; error 30 us; skew 500 ppm
I20260811 09:07:17.606774 3371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:17.610074 3371 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260811 09:07:17.611588 3371 webserver.cc:460] Webserver started at http://127.9.40.125:44851/ using document root <none> and password file <none>
I20260811 09:07:17.612315 3371 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:17.612450 3371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:17.612839 3371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:17.615986 3371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data/instance:
uuid: "0992d01d7bd748ef910d6bf7f6cd30f3"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "a2bb2d08a0312f74dcb7914775da93c6"
server_key_iv: "2f89a89ca92a37fc6e73095bb0d7aafd"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.616880 3371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal/instance:
uuid: "0992d01d7bd748ef910d6bf7f6cd30f3"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "a2bb2d08a0312f74dcb7914775da93c6"
server_key_iv: "2f89a89ca92a37fc6e73095bb0d7aafd"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.622391 3371 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:07:17.625988 3389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:17.627332 3371 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 09:07:17.627517 3371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal
uuid: "0992d01d7bd748ef910d6bf7f6cd30f3"
format_stamp: "Formatted at 2026-08-11 09:07:17 on dist-test-slave-cbsf"
server_key: "a2bb2d08a0312f74dcb7914775da93c6"
server_key_iv: "2f89a89ca92a37fc6e73095bb0d7aafd"
server_key_version: "encryptionkey@0"
I20260811 09:07:17.627739 3371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:17.650250 3371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:17.654477 3371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:17.654776 3371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:17.674757 3371 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:41843
I20260811 09:07:17.674777 3448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:41843 every 8 connection(s)
I20260811 09:07:17.676169 3371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/data/info.pb
I20260811 09:07:17.677270 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 3371
I20260811 09:07:17.677520 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-1/wal/instance
I20260811 09:07:17.678081 9377 external_mini_cluster.cc:1428] Setting key 889107228a1b055ef69dbb6d5ff0b9ec
I20260811 09:07:17.681320 3449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:17.692065 3449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
I20260811 09:07:17.715719 3452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.695819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44092 (local address 127.9.40.126:43033)
0811 09:07:17.696362 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:17.696367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:17.697654 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 09:07:17.699667 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:17.699672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:17.699786 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:17.699915 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:17.701619 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.702353 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.703937 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.704359 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.704503 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.708871 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:17.708877 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:17.708881 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:17.708930 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:17.711815 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.712910 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.712914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.712916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.713057 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.713648 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.713651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.713653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.713945 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:17.714072 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:17.715143 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0811 09:07:17.715413 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":320,"threads_started":1}
I20260811 09:07:17.716939 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.696550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43033 (local address 127.0.0.1:44092)
0811 09:07:17.697449 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.697517 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:17.699343 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:17.700173 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:17.700181 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:17.700818 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:17.701395 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.701412 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.702540 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.702544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.703784 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.703796 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.704625 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.704632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.704935 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.705707 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:17.705732 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:17.708502 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:17.712002 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.712012 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.712035 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.712769 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.713182 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.713189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.713196 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.713518 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.714102 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:17.714113 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:17.714653 (+ 540us) client_negotiation.cc:769] Sending connection context
0811 09:07:17.714889 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 09:07:17.715413 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":380,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
I20260811 09:07:17.718948 3449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:17.736618 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.720214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60068 (local address 127.9.40.125:41843)
0811 09:07:17.720978 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:17.720989 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:17.721024 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:07:17.721082 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:17.721092 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:17.721342 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:17.721494 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:17.722214 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.723920 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.725186 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.726094 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.726249 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.729517 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:17.729534 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:17.729554 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:17.729600 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:17.733481 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.734293 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.734299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.734301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.734414 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.734853 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.734859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.734861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.735293 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:17.735492 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:17.735970 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 09:07:17.736316 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":124,"server-negotiator.queue_time_us":564,"thread_start_us":380,"threads_started":1}
I20260811 09:07:17.736738 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.719918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60068)
0811 09:07:17.720113 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.720133 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:17.720235 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:17.721517 (+ 1282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:17.721520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:17.721887 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:17.722038 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.722046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.724081 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.724085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.725045 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.725056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.726240 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.726244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.726356 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.726962 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:17.726978 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:17.729292 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:17.733611 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.733616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.733620 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.734159 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.734526 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.734532 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.734535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.734709 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.735433 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:17.735440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:17.735628 (+ 188us) client_negotiation.cc:769] Sending connection context
0811 09:07:17.735787 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 09:07:17.736194 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:07:17.739483 3449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:17.740659 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.740207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37164)
0811 09:07:17.740388 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.740501 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:07:17.740978 3394 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.742615 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.742268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37172)
0811 09:07:17.742417 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.742505 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:07:17.743146 3449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260811 09:07:17.761402 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
/tmp/dist-test-taskSMwUEY/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:42593
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:39169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:07:17.792497 3449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 1
I20260811 09:07:17.793890 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.793420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37182)
0811 09:07:17.793628 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.793736 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:07:17.794940 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.794588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37196)
0811 09:07:17.794751 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.794835 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:07:17.795164 3449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.825133 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } attempt: 5
Aug 11 09:07:17 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:17.840175 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.826185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.9.40.125:41843)
0811 09:07:17.826367 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:17.826372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:17.826408 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:07:17.826482 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:17.826486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:17.826549 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:17.826636 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:17.827884 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.828575 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.830438 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.830906 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.831007 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.834868 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:17.834873 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:17.834876 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:17.834909 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:17.837585 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.838387 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.838390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.838392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.838511 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:17.838914 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:17.838917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:17.838919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:17.839206 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:17.839314 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:17.839775 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 09:07:17.839954 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:07:17.840384 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.825859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60084)
0811 09:07:17.826143 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.826172 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:17.826333 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:17.826727 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:17.826734 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:17.827397 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:17.827696 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.827716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.828778 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.828786 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.830290 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:17.830308 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:17.831067 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:17.831073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:17.831341 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:17.832006 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:17.832030 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:17.834670 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:17.837738 (+ 3068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.837744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.837748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.838276 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.838621 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:17.838625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:17.838629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:17.838805 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:17.839327 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:17.839332 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:17.839511 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:07:17.839689 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 09:07:17.840021 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 09:07:17.841681 3363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:17.842633 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.842229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37200)
0811 09:07:17.842410 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.842486 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 09:07:17.843498 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.843181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37210)
0811 09:07:17.843328 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.843408 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:07:17.843713 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.863389 3449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 2
I20260811 09:07:17.864517 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.864066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37212)
0811 09:07:17.864262 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.864360 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 09:07:17.865578 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.865245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37226)
0811 09:07:17.865404 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.865468 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:07:17.865785 3449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.894922 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 1
I20260811 09:07:17.896051 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.895570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37232)
0811 09:07:17.895781 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.895883 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 09:07:17.897075 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.896739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37242)
0811 09:07:17.896896 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.896974 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:07:17.897297 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.956990 3449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 3
I20260811 09:07:17.958247 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.957767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37248)
0811 09:07:17.957980 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.958073 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:07:17.959348 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.958927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37250)
0811 09:07:17.959169 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.959241 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260811 09:07:17.959563 3449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:17.965495 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 2
I20260811 09:07:17.966506 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.966059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37258)
0811 09:07:17.966284 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.966361 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:07:17.967453 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:17.967126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37266)
0811 09:07:17.967298 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:17.967358 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:07:17.967643 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:18.073859 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 3
I20260811 09:07:18.075129 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.074650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37268)
0811 09:07:18.074860 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.074953 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:07:18.076267 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.075858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37278)
0811 09:07:18.076087 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.076158 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 09:07:18.076462 3363 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
I20260811 09:07:18.102869 3449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 4
I20260811 09:07:18.104079 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.103598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37292)
0811 09:07:18.103845 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.103931 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:07:18.105171 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.104833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37304)
0811 09:07:18.105002 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.105065 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:07:18.105362 3449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:42593: Network error: Client connection negotiation failed: client connection to 127.9.40.124:42593: connect: Connection refused (error 111)
W20260811 09:07:18.130853 3457 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:07:18.131258 3457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:18.131402 3457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:18.131539 3457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:18.139482 3457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:18.139657 3457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:18.139802 3457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:18.139895 3457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:18.140018 3457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:18.150592 3457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:39169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43033,127.9.40.125:41843,127.9.40.124:42593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:18.152022 3457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:18.153513 3457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:18.165498 3467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:18.165584 3465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:18.165606 3463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:18.166563 3457 server_base.cc:1034] running on GCE node
I20260811 09:07:18.167323 3457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:18.168789 3457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:18.170047 3457 hybrid_clock.cc:648] HybridClock initialized: now 1786439238169987 us; error 48 us; skew 500 ppm
I20260811 09:07:18.170414 3457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:18.173753 3457 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260811 09:07:18.175284 3457 webserver.cc:460] Webserver started at http://127.9.40.124:45243/ using document root <none> and password file <none>
I20260811 09:07:18.176020 3457 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:18.176160 3457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:18.176508 3457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:18.179757 3457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data/instance:
uuid: "908896580ce9412a9f7292cbafa258c2"
format_stamp: "Formatted at 2026-08-11 09:07:18 on dist-test-slave-cbsf"
server_key: "c300ebb2ef217668c6bacea6dd99f50e"
server_key_iv: "9634b990a6df5b6c8ed6b6c98c2010b7"
server_key_version: "encryptionkey@0"
I20260811 09:07:18.180585 3457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal/instance:
uuid: "908896580ce9412a9f7292cbafa258c2"
format_stamp: "Formatted at 2026-08-11 09:07:18 on dist-test-slave-cbsf"
server_key: "c300ebb2ef217668c6bacea6dd99f50e"
server_key_iv: "9634b990a6df5b6c8ed6b6c98c2010b7"
server_key_version: "encryptionkey@0"
I20260811 09:07:18.185863 3457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:07:18.189391 3476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:18.190729 3457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:07:18.190943 3457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal
uuid: "908896580ce9412a9f7292cbafa258c2"
format_stamp: "Formatted at 2026-08-11 09:07:18 on dist-test-slave-cbsf"
server_key: "c300ebb2ef217668c6bacea6dd99f50e"
server_key_iv: "9634b990a6df5b6c8ed6b6c98c2010b7"
server_key_version: "encryptionkey@0"
I20260811 09:07:18.191175 3457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:18.203254 3457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:18.207674 3457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:18.207952 3457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:18.227396 3457 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:42593
I20260811 09:07:18.227403 3539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:42593 every 8 connection(s)
I20260811 09:07:18.228888 3457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/data/info.pb
I20260811 09:07:18.232743 3540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:18.238942 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu as pid 3457
I20260811 09:07:18.239171 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskSMwUEY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786438852478101-9377-0/minicluster-data/master-2/wal/instance
I20260811 09:07:18.239729 9377 external_mini_cluster.cc:1428] Setting key e92ac198c50b5c42ec90e48cf7b3df24
I20260811 09:07:18.239797 3540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:18.253692 3363 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 4
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:07:18.262176 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.242762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44096 (local address 127.9.40.126:43033)
0811 09:07:18.243314 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.243330 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.244279 (+ 949us) server_negotiation.cc:407] Connection header received
0811 09:07:18.245445 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.245449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.245565 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.245728 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.247081 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.247831 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.249615 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.250194 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.250287 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.254290 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.254297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.254302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.254359 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.257732 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.258996 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.259006 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.259010 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.259195 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.259733 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.259740 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.259745 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.260144 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.260300 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.261473 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.261730 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260811 09:07:18.263070 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.242725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43033 (local address 127.0.0.1:44096)
0811 09:07:18.244097 (+ 1372us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.244159 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:18.245141 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:18.245819 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:18.245826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:18.246522 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:18.246898 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.246916 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.247998 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.248005 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.249454 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.249471 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.250416 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.250423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.250649 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.251477 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:18.251504 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:18.253998 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:18.258065 (+ 4067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.258075 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.258085 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.258845 (+ 760us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.259327 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.259332 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.259336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.259575 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.260290 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:18.260300 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:18.260792 (+ 492us) client_negotiation.cc:769] Sending connection context
0811 09:07:18.261076 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 09:07:18.261602 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"thread_start_us":937,"threads_started":1}
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:07:18.263870 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.241641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.9.40.124:42593)
0811 09:07:18.242522 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.242533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.242573 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:07:18.245141 (+ 2568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.245170 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.245473 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.245651 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.247133 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.248560 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.250014 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.250714 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.250859 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.255859 (+ 5000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.255873 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.255895 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.255944 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.259812 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.260667 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.260678 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.260682 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.260874 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.261728 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.261734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.261738 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.262223 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.262503 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.263171 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.263426 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":466,"threads_started":1}
I20260811 09:07:18.265211 3540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:18.271342 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.254362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37312)
0811 09:07:18.254680 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.254699 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:18.254799 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:18.255707 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:18.255716 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:18.256299 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:18.256522 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.256536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.258392 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.258397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.259767 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.259781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.260595 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.260602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.260856 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.261446 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:18.261463 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:18.264401 (+ 2938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:18.268467 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.268473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.268477 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.269059 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.269476 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.269481 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.269485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.269663 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.270316 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:18.270322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:18.270490 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 09:07:18.270645 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:07:18.270945 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 09:07:18.271397 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.254582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37312 (local address 127.9.40.124:42593)
0811 09:07:18.255371 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.255378 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.255402 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:07:18.255449 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.255453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.255523 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.255650 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.256744 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.258243 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.259909 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.260435 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.260556 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.264649 (+ 4093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.264654 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.264657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.264685 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.268338 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.269214 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.269219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.269223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.269358 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.269794 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.269801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.269804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.270160 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.270322 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.270765 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.270976 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":713,"threads_started":1}
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260811 09:07:18.278899 3363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: Bootstrap starting.
I20260811 09:07:18.281306 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.266032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:41843 (local address 127.0.0.1:60086)
0811 09:07:18.266229 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.266247 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:18.266346 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:18.267554 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:18.267558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:18.268136 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:18.268369 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.268383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.269877 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.269885 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.271176 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.271190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.272257 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.272263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.272445 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.273002 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:18.273025 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:18.275038 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:18.278417 (+ 3379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.278441 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.278445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.278998 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.279506 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.279514 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.279520 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.279694 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.280236 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:18.280241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:18.280417 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 09:07:18.280584 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 09:07:18.280903 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 09:07:18.281711 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.266891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.9.40.125:41843)
0811 09:07:18.267097 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.267103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.267123 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:07:18.267183 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.267188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.267273 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.267372 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.268615 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.269688 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.271352 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.271905 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.272008 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.275236 (+ 3228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.275241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.275244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.275276 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.278255 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.279196 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.279201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.279205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.279373 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.279818 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.279824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.279827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.280113 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.280273 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.280782 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.281038 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:07:18.282845 3540 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:07:18.283861 3363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:18.285415 3363 log.cc:824] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:18.286299 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.267350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.9.40.126:43033)
0811 09:07:18.267508 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.267516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.267533 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:18.267580 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.267585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.267668 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.267781 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.269306 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.270184 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.272035 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.272593 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.273025 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.278907 (+ 5882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.278914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.278923 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.278963 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.283057 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.283899 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.283906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.283910 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.284068 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.284571 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.284579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.284582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.285021 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.285367 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.285739 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.285971 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:07:18.290838 3363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: No bootstrap required, opened a new log
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439238, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260811 09:07:18.299162 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.283598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37320)
0811 09:07:18.283787 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.283810 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:18.283922 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:18.284331 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:18.284335 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:18.284882 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:18.285079 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.285093 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.286615 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.286620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.288046 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.288061 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.288897 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.288902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.289101 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.289618 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:18.289634 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:18.291801 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:18.295902 (+ 4101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.295909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.295914 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.296514 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.297002 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.297009 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.297013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.297214 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.298006 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:18.298013 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:18.298191 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 09:07:18.298365 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 09:07:18.298709 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 09:07:18.299216 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.283836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.9.40.124:42593)
0811 09:07:18.284035 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.284040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.284057 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:18.284104 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.284111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.284202 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.284313 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.285232 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.286456 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.288227 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.288763 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.288891 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.292013 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.292020 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.292025 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.292062 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.295734 (+ 3672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.296660 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.296666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.296669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.296867 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.297346 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.297351 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.297354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.297762 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.297907 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.298510 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.298856 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:07:18.301875 3363 raft_consensus.cc:348] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.302294 3363 raft_consensus.cc:729] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de2452320b5e4e88935df891f20c6d8b, State: Initialized, Role: FOLLOWER
I20260811 09:07:18.302827 3363 consensus_queue.cc:260] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.305446 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.288438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60092 (local address 127.9.40.125:41843)
0811 09:07:18.288600 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.288605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.288624 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:18.288680 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.288686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.288801 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.288890 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.291058 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.292099 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.293588 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.294164 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.294282 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.299789 (+ 5507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.299794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.299797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.299831 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.302768 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.303456 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.303466 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.303468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.303602 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.303989 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.303994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.303997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.304273 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.304558 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.304885 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.305121 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 09:07:18.306558 3540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: Bootstrap starting.
I20260811 09:07:18.310202 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:07:18.312076 3540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:18.312572 3334 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "de2452320b5e4e88935df891f20c6d8b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44108
I20260811 09:07:18.313355 3550 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.313829 3550 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.313796 3540 log.cc:824] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:18.314818 3363 sys_catalog.cc:564] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:18.314955 3334 raft_consensus.cc:482] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:07:18.315199 3334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:18.317480 3540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: No bootstrap required, opened a new log
I20260811 09:07:18.319363 3334 raft_consensus.cc:504] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.322950 3334 leader_election.cc:288] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [CANDIDATE]: Term 1 election: Requested vote from peers 0992d01d7bd748ef910d6bf7f6cd30f3 (127.9.40.125:41843), 908896580ce9412a9f7292cbafa258c2 (127.9.40.124:42593)
I20260811 09:07:18.329092 3540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.329751 3540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908896580ce9412a9f7292cbafa258c2, State: Initialized, Role: FOLLOWER
I20260811 09:07:18.330415 3540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
W20260811 09:07:18.353348 3307 tablet.cc:2369] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:07:18.363826 3572 catalog_manager.cc:1558] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:07:18.364243 3572 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:07:18.376130 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.376626 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.335438 3282 master_runner.cc:186] Error getting master registration for 127.9.40.125:41843: OK, instance_id { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" instance_seqno: 1786439237669225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:07:18.379417 3282 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:07:18.377763 3540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:18.377856 3513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de2452320b5e4e88935df891f20c6d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "908896580ce9412a9f7292cbafa258c2"
I20260811 09:07:18.382567 3513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:18.386328 3513 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2452320b5e4e88935df891f20c6d8b in term 1.
I20260811 09:07:18.387081 3303 leader_election.cc:302] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 908896580ce9412a9f7292cbafa258c2, de2452320b5e4e88935df891f20c6d8b; no voters:
I20260811 09:07:18.388896 3550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:18.396389 3454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.331107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.9.40.125:41843)
0811 09:07:18.331383 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.331392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.357380 (+ 25988us) server_negotiation.cc:407] Connection header received
0811 09:07:18.359725 (+ 2345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.359731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.359821 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.359897 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.370389 (+ 10492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.371921 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.381028 (+ 9107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.381822 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.381949 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.387239 (+ 5290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.387258 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.387266 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.387315 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.391197 (+ 3882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.392892 (+ 1695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.392900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.392904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.393091 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.393952 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.393958 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.393962 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.394406 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.394675 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.395634 (+ 959us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.395983 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260811 09:07:18.404556 3449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } attempt: 5
I20260811 09:07:18.404903 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.329631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44122 (local address 127.9.40.126:43033)
0811 09:07:18.329835 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.329841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.354857 (+ 25016us) server_negotiation.cc:407] Connection header received
0811 09:07:18.355521 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.355531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.355664 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.355759 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.364216 (+ 8457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.365949 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.376880 (+ 10931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.377690 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.377849 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.383863 (+ 6014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.383871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.383876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.383925 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.390085 (+ 6160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.397778 (+ 7693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.397788 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.397794 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.398020 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.400943 (+ 2923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.400953 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.400959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.401797 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.402188 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.403918 (+ 1730us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.404268 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 09:07:18 dist-test-slave-cbsf krb5kdc[3269](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439237, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260811 09:07:18.428001 3453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.405227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:42593 (local address 127.0.0.1:37342)
0811 09:07:18.405670 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:18.405696 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:18.405838 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:18.406500 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:18.406508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:18.407227 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:18.407520 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.407537 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.409592 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.409612 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.411571 (+ 1959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:18.411592 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.413321 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.413330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:18.413694 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.414590 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:18.414614 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:18.418722 (+ 4108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:18.423929 (+ 5207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.423934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.423939 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.424536 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.425388 (+ 852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:18.425393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:18.425397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:18.425576 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:18.426862 (+ 1286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:18.426869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:18.427055 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 09:07:18.427229 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 09:07:18.427596 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254}
I20260811 09:07:18.430116 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.405381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.9.40.124:42593)
0811 09:07:18.405589 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.405595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.405966 (+ 371us) server_negotiation.cc:407] Connection header received
0811 09:07:18.406049 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.406058 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.406162 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.406270 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.407851 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.409373 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.411819 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.412854 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.413038 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.419053 (+ 6015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.419065 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.419073 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.419132 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.423676 (+ 4544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.424835 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.424841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.424844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.425162 (+ 318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.425811 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.425819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.425823 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.426661 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.428520 (+ 1859us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.428791 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.429618 (+ 827us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:07:18.433671 3550 raft_consensus.cc:686] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [term 1 LEADER]: Becoming Leader. State: Replica: de2452320b5e4e88935df891f20c6d8b, State: Running, Role: LEADER
I20260811 09:07:18.434507 3550 consensus_queue.cc:237] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.436338 3449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: Bootstrap starting.
I20260811 09:07:18.440968 3449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:18.441550 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:18.363378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.9.40.124:42593)
0811 09:07:18.363636 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.363645 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.369876 (+ 6231us) server_negotiation.cc:407] Connection header received
0811 09:07:18.370071 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.370078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.370206 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.370327 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.379977 (+ 9650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.381095 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.384822 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:18.385650 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:18.385765 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:18.424048 (+ 38283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:18.424058 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:18.424066 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:18.424134 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:18.429328 (+ 5194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.437896 (+ 8568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.437906 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.437913 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.438181 (+ 268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:18.438870 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:18.438879 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:18.438885 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:18.439380 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:18.439799 (+ 419us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:18.440578 (+ 779us) server_negotiation.cc:299] Negotiation successful
0811 09:07:18.440992 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:07:18.445662 3449 log.cc:824] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:18.449877 3579 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2452320b5e4e88935df891f20c6d8b. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.454223 3579 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:18.455590 3449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: No bootstrap required, opened a new log
W20260811 09:07:18.460009 3485 tablet.cc:2369] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:18.471437 3585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:18.477773 3585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:18.478623 3449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.479067 3449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0992d01d7bd748ef910d6bf7f6cd30f3, State: Initialized, Role: FOLLOWER
I20260811 09:07:18.479638 3449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } }
I20260811 09:07:18.486559 3590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.487020 3590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.489831 3449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:18.489923 3421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de2452320b5e4e88935df891f20c6d8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3"
I20260811 09:07:18.490586 3421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:18.495698 3421 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de2452320b5e4e88935df891f20c6d8b in term 1.
I20260811 09:07:18.506464 3598 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:07:18.507097 3513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [term 1 FOLLOWER]: Refusing update from remote peer de2452320b5e4e88935df891f20c6d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:18.507759 3421 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [term 1 FOLLOWER]: Refusing update from remote peer de2452320b5e4e88935df891f20c6d8b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:18.508309 3579 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:18.508682 3550 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:18.530429 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2452320b5e4e88935df891f20c6d8b. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.530789 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.536326 3601 mvcc.cc:205] Tried to move back new op lower bound from 7317255120910065664 to 7317255120635273216. Current Snapshot: MvccSnapshot[applied={T|T < 7317255120910065664}]
I20260811 09:07:18.543672 3585 catalog_manager.cc:1347] Generated new cluster ID: 97096ee26bb840d0ae1a73965d3370e3
I20260811 09:07:18.543807 3585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:18.546008 3579 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.546384 3579 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:18.546824 3592 mvcc.cc:205] Tried to move back new op lower bound from 7317255120910065664 to 7317255120635273216. Current Snapshot: MvccSnapshot[applied={T|T < 7317255120910065664 or (T in {7317255120910065664})}]
I20260811 09:07:18.549279 3575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.549510 3575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.519797 3457 master_runner.cc:186] Error getting master registration for 127.9.40.125:41843: OK, instance_id { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" instance_seqno: 1786439237669225 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:07:18.553362 3457 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:07:18.555827 3590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de2452320b5e4e88935df891f20c6d8b. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.556156 3590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.558162 3550 sys_catalog.cc:455] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.558559 3550 sys_catalog.cc:458] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:18.564989 3610 mvcc.cc:205] Tried to move back new op lower bound from 7317255120910065664 to 7317255120635273216. Current Snapshot: MvccSnapshot[applied={T|T < 7317255120910065664}]
I20260811 09:07:18.572476 3590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de2452320b5e4e88935df891f20c6d8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de2452320b5e4e88935df891f20c6d8b" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43033 } } peers { permanent_uuid: "0992d01d7bd748ef910d6bf7f6cd30f3" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 41843 } } peers { permanent_uuid: "908896580ce9412a9f7292cbafa258c2" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 42593 } } }
I20260811 09:07:18.572865 3590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:07:18.589709 3585 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:18.591708 3585 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:18.597538 3614 catalog_manager.cc:1259] Loaded cluster ID: 97096ee26bb840d0ae1a73965d3370e3
I20260811 09:07:18.597661 3614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: loading cluster ID for follower catalog manager: success
I20260811 09:07:18.600147 3614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: acquiring CA information for follower catalog manager: success
W20260811 09:07:18.601534 3614 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 09:07:18.615114 3585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de2452320b5e4e88935df891f20c6d8b: Generated new TSK 0
I20260811 09:07:18.616441 3585 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:07:18.732188 3546 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.717380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.9.40.126:43033)
0811 09:07:18.717629 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.717637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.729583 (+ 11946us) server_negotiation.cc:407] Connection header received
0811 09:07:18.730401 (+ 818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.730406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.730512 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.730605 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.731890 (+ 1285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44124: BlockingRecv error: recv got EOF from 127.0.0.1:44124 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 09:07:18.732328 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.717460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60104 (local address 127.9.40.125:41843)
0811 09:07:18.717645 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.717654 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.729791 (+ 12137us) server_negotiation.cc:407] Connection header received
0811 09:07:18.729962 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.729970 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.730066 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.730207 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.732065 (+ 1858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60104: BlockingRecv error: recv got EOF from 127.0.0.1:60104 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 09:07:18.735466 3548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.730496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37344 (local address 127.9.40.124:42593)
0811 09:07:18.730739 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.730748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.733307 (+ 2559us) server_negotiation.cc:407] Connection header received
0811 09:07:18.733595 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.733604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.733709 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.733802 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.735112 (+ 1310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37344: BlockingRecv error: recv got EOF from 127.0.0.1:37344 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 09:07:18.743639 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.741372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60116 (local address 127.9.40.125:41843)
0811 09:07:18.741608 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.741616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.742111 (+ 495us) server_negotiation.cc:407] Connection header received
0811 09:07:18.742435 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.742439 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.742563 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.742680 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.743396 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60116: BlockingRecv error: recv got EOF from 127.0.0.1:60116 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 09:07:18.751435 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.749265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.9.40.125:41843)
0811 09:07:18.749496 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.749505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.750023 (+ 518us) server_negotiation.cc:407] Connection header received
0811 09:07:18.750255 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.750259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.750335 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.750447 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.751199 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60120: BlockingRecv error: recv got EOF from 127.0.0.1:60120 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:07:18.758988 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.756794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60124 (local address 127.9.40.125:41843)
0811 09:07:18.757009 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.757018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.757558 (+ 540us) server_negotiation.cc:407] Connection header received
0811 09:07:18.757765 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.757771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.757872 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.757986 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.758725 (+ 739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60124: BlockingRecv error: recv got EOF from 127.0.0.1:60124 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 09:07:18.768254 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.766031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60138 (local address 127.9.40.125:41843)
0811 09:07:18.766326 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.766335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.766884 (+ 549us) server_negotiation.cc:407] Connection header received
0811 09:07:18.767102 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.767107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.767198 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.767309 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.768022 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60138: BlockingRecv error: recv got EOF from 127.0.0.1:60138 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
W20260811 09:07:18.782157 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.779825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.9.40.125:41843)
0811 09:07:18.780066 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.780075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.780633 (+ 558us) server_negotiation.cc:407] Connection header received
0811 09:07:18.780906 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.780912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.780996 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.781105 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.781891 (+ 786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60146: BlockingRecv error: recv got EOF from 127.0.0.1:60146 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 09:07:18.803892 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.801606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.9.40.125:41843)
0811 09:07:18.801860 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.801870 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.802416 (+ 546us) server_negotiation.cc:407] Connection header received
0811 09:07:18.802652 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.802657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.802735 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.802866 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.803618 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60148: BlockingRecv error: recv got EOF from 127.0.0.1:60148 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 09:07:18.841835 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.839412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60154 (local address 127.9.40.125:41843)
0811 09:07:18.839676 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.839686 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.840267 (+ 581us) server_negotiation.cc:407] Connection header received
0811 09:07:18.840503 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.840507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.840585 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.840702 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.841539 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60154: BlockingRecv error: recv got EOF from 127.0.0.1:60154 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:07:18.913944 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:18.911391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60158 (local address 127.9.40.125:41843)
0811 09:07:18.911637 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:18.911645 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:18.912240 (+ 595us) server_negotiation.cc:407] Connection header received
0811 09:07:18.912464 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:18.912471 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:18.912579 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:18.912792 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:18.913703 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60158: BlockingRecv error: recv got EOF from 127.0.0.1:60158 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 09:07:19.048470 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:19.046402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60174 (local address 127.9.40.125:41843)
0811 09:07:19.046617 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.046626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.046962 (+ 336us) server_negotiation.cc:407] Connection header received
0811 09:07:19.047197 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.047202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.047274 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.047382 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:19.048186 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60174: BlockingRecv error: recv got EOF from 127.0.0.1:60174 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 09:07:19.309991 3454 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:19.308217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60186 (local address 127.9.40.125:41843)
0811 09:07:19.308469 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.308474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.308654 (+ 180us) server_negotiation.cc:407] Connection header received
0811 09:07:19.308856 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.308860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.308941 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.309046 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:19.309810 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60186: BlockingRecv error: recv got EOF from 127.0.0.1:60186 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:07:19.511113 3598 catalog_manager.cc:1259] Loaded cluster ID: 97096ee26bb840d0ae1a73965d3370e3
I20260811 09:07:19.511265 3598 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: loading cluster ID for follower catalog manager: success
I20260811 09:07:19.514016 3598 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: acquiring CA information for follower catalog manager: success
I20260811 09:07:19.516098 3598 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 908896580ce9412a9f7292cbafa258c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:07:19.604250 3614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0992d01d7bd748ef910d6bf7f6cd30f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:07:19.834522 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:19.826919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60188 (local address 127.9.40.125:41843)
0811 09:07:19.827698 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.827704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.827722 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:07:19.827788 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.827792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.827946 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.828189 (+ 243us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:19.828778 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.830063 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.832113 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.832511 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.832614 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:19.833265 (+ 651us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:19.833830 (+ 565us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:19.833979 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:19.834149 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 09:07:19.834221 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":560,"threads_started":1}
I20260811 09:07:19.848042 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:19.840746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.9.40.124:42593)
0811 09:07:19.841395 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.841401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.841672 (+ 271us) server_negotiation.cc:407] Connection header received
0811 09:07:19.841858 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.841864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.842063 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.842250 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:19.842597 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.843811 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.845224 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.845917 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.846045 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:19.846555 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:19.847309 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:19.847453 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:19.847563 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 09:07:19.847649 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":307,"threads_started":1}
I20260811 09:07:19.864192 3693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:19.854716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60194 (local address 127.9.40.125:41843)
0811 09:07:19.854961 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.854971 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.857424 (+ 2453us) server_negotiation.cc:407] Connection header received
0811 09:07:19.857524 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.857533 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.857645 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.857748 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:19.858758 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.859660 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.861953 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.862550 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.862661 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:19.863393 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:19.863528 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:19.863639 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:19.863797 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 09:07:19.863895 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:07:19.864254 3699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:19.857571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37366 (local address 127.9.40.124:42593)
0811 09:07:19.857762 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.857769 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.858449 (+ 680us) server_negotiation.cc:407] Connection header received
0811 09:07:19.858645 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.858652 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.858761 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.858858 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:19.859407 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.860251 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.862034 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.862662 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.862772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:19.863464 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:19.863580 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:19.863662 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:19.863899 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 09:07:19.863977 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:07:19.864660 3707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:19.854286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44132 (local address 127.9.40.126:43033)
0811 09:07:19.856880 (+ 2594us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:19.856894 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:19.856911 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:19.857004 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:19.857008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:19.857191 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:19.857405 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:19.858332 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.859494 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.861948 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:19.862398 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:19.862548 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:19.863221 (+ 673us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:19.863698 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:19.863801 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:19.864249 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 09:07:19.864344 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2454,"thread_start_us":2304,"threads_started":1}
I20260811 09:07:19.869616 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 3282
I20260811 09:07:19.870711 3357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:19.986162 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 3282
I20260811 09:07:20.015586 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 3371
I20260811 09:07:20.016876 3443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:20.126828 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 3371
I20260811 09:07:20.152774 9377 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu pid 3457
I20260811 09:07:20.153964 3535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:20.264546 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskSMwUEY/build/asan/bin/kudu with pid 3457
2026-08-11T09:07:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5234 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10524 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (387754 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST